Saravanan666/pypi-org huggingface dataset:
Package: llmfast
LLMFASTllmfast is your go-to Python library for simplifying the development of Large Language Model (LLM) applications. With just a few lines of code, you can harness the power of LLMs, saving you time and reducing complexity. Our library offers effortless integration, minimal coding requirements, and full customization to meet your unique needs. It's designed for optimized performance, ensuring smooth application execution, even with extensive language model usage. Explore our comprehensive documentation and examples to kickstart your LLM application development journey with ease. Unlock the potential of LLMs today with llmfast!ChatbotImport theChatbotclass: In your Python script, import theChatbotclass fromllmfast. You can do this by adding the following line at the beginning of your code:fromllmfastimportChatbotObtain an OpenAI API key: To use thellmfastlibrary, you'll need an API key from OpenAI. If you don't have one, sign up on the OpenAI website and obtain an API key.Create aChatbotinstance: Initialize the Chatbot class with your OpenAI API key and specify the desired role for your chatbot. Here's an example of creating aChatbotinstance:Replace"YOUR-OPENAI-API-KEY"with your actual OpenAI API key, and"ENTER-YOUR-CHATBOT-ROLE"with the desired role for your chatbot.Deploy the chatbot: Set thedeployparameter toTruewhen creating theChatbotinstance. This will deploy the chatbot and make it available for use.To deploy or terminate the deployment of a chatbot created using themlfastlibrary, you can set thedeployparameter toTrueorFalsewhen creating theChatbotinstance, respectively.Chatbot(api_key="YOUR-OPENAI-API-KEY",role="ENTER-YOUR-CHATBOT-ROLE",deploy=True)
Package: llm-few-shot-gen
LLM Few Shot Gen[!WARNING]This library is not maintained anymore. Checkout the new project llm-prompting-gen.Installationpip install llm-few-shot-genWhat is LLM Few Shot Generator?Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. However, fine-tune them is not always possible or too expansive. In-context learning like Few Shot learning tries to solve this problem. Based on a few number of examples within the prompt a specific output can be obtained. This llm-few-shot-gen library enables few shot learning in a convenient way.How can I use it?The core functionality is thegeneratorsmodule. The classPromptEngineeringGeneratordefines all requirements for a LLM to generate text based on prompt engineering techniques. If you want your output to be parsed into an pydantic dataclass checkout the classParsablePromptEngineeringGenerator.How to customise the class for my own use case?The classPromptEngineeringGeneratorcontains two core parts 1. LLM 2. prompt engineering dataclass. If you want to initialise the generator class for you custom use case, you need to define a prompt engineering JSON file matching the format ofllm_few_shot_gen.models.prompt_engineering.PromptElements.The JSON file can contain the following prompt elements in any combination tailored to your use case:Role:
The role in which the LLM should respondInstruction:
The task of the LLMContext:
Context with relevant information to solve the taskOutput Format:
Description how the LLM output format should look likeFew Shot Examples:
Few shot examples with optional introductionInput:
Target which the LLM should execute the task on. Could be for example a user question, or a text block to summarize.ShowcasesNotebook Showcase: Midjourney PromptNotebook Showcase: Keyword ExtractorApp: Image Gen AI Prompt GeneratorLicensellm-few-shot-genis distributed under the terms of theMITlicense.
Package: llmfn
llmfnllmfnis a Python library to approximate a function using OpenAI's API. You can use it to easily train a language model to approximate your own functions with few-shot prompting.InstallationYou can install the package from pip:pip install llmfnUsageFirst, you need to set your OpenAI API key:importosimportopenaiopenai.api_key=os.getenv("OPENAI_API_KEY")Then, you can define a list of examples of the function's behavior:fromllmfnimportArgumentsfromllmfnimportFunctionExampleexamples=[FunctionExample(arguments=Arguments.call(2,3),output=5),FunctionExample(arguments=Arguments.call(5,7),output=12),# ...]Finally, you can use the llmfn decorator to create an approximated version of your function:fromllmfnimportllmfn@llmfn(examples=examples,function_name="my_function")defmy_function(a:int,b:int)->int:returna+bassertmy_function(2,3)==5Alternatively, you can use themake_llmfnfunction to create an approximated version of your function without using the decorator:fromllmfnimportmake_llmfnblackbox=make_llmfn(examples=examples,function_name="my_function")assertblackbox(2,3)==5Advanced UsageChanging the DecoderBy default, the decoder is set tolambda x: x, which simply returns the output as a string. You can change the decoder to parse the output into a different data type:fromllmfnimportmake_llmfndefdecoder(output:str)->int:returnint(output)blackbox=make_llmfn(examples=examples,function_name="my_function",decoder=decoder)assertblackbox(2,3)==5The most useful decoder (and the most dangerous) iseval, which will evaluate the output as Python code:fromllmfnimportmake_llmfnblackbox=make_llmfn(examples=examples,function_name="my_function",decoder=eval)assertblackbox(2,3)==5Use this with caution - it could be used to execute arbitrary code. (This is why it's not the default decoder.)Changing the EngineBy default, the engine is set totext-davinci-003. You can change the engine to a different OpenAI engine:fromllmfnimportmake_llmfnblackbox=make_llmfn(examples=examples,function_name="my_function",engine="text-curie-001")assertblackbox(2,3)==5LimitationsThe function's output must be a string.The approximated function can only handle arguments that are compatible with the examples.The approximated function may not work with complex or large functions.The API usage may be subject to rate limits and other restrictions imposed by OpenAI.ContributingWe welcome contributions to this project. If you have any ideas or suggestions, please open an issue or submit a pull request.LicenseThis project is licensed under the MIT License.
Package: llmformat
LLM FormatIntroductionLLM Format is a Large-language model formatter that constrain the outputs of language model to follow certain rules. Our design is to make this tool flexible and efficient to adapt to different use cases.We currently supportsvllmand anyLALR(1) grammarincludingJSONformat.Different from other packages including lm-format-enforcer, jsonformer and guidance. This package ensures the generated text from LLM to be sound and complete.TutorialInstallationpip install llmformatUsageTo enforce a new type of grammar, a grammar file written in EBNF is needed. We provide the JSON example ashere.Once it is written, we can only need one-line code change to enforce the generation.In vllm, add this option to sampling_param.sampling_param.logits_processors=[llmformat.llminterface.build_vllm_logits_processor(model, "/root/llmformat/llmformat/json_min.bnf")]ExampleThe example of working on Llama2 and vllm can be foundhere. Note that you may want to change the location of grammar file.Add support for customized JSONAdd more integrationsKnown IssuesWe use cache to accelerate grammar parsing. The speed will becomes faster as it runns longer.
Package: llmfuncs
llmfuncsllmfuncsis a Python package for documenting, managing, and validating Python functions based on their signature and docstrings. It automatically generates JSON Schema for functions, and provides a way to validate function arguments against the schema.FeaturesAutomatic generation of JSON Schema from Python function signature and docstrings.Support for Python built-in types and many typing module types.Use and validate functions with JSON arguments against their schema.Management of multiple functions through a ToolCollection object.Bulk addition of functions from a module, a package, or a glob pattern.Simple and intuitive API.InstallationInstallllmfuncsusing pip:pipinstallllmfuncsUsageToolTheToolclass encapsulates a function and its associated schema.fromllmfuncs.toolimportTooldefgreet(name:str)->str:"""Greet someone.Args:name: Name of the person to greet.Returns:A greeting message."""returnf"Hello,{name}!"tool=Tool(greet)print(tool.name())# "greet"print(tool.schema())ToolCollectionTheToolCollectionclass is a container for multipleToolobjects. It provides methods to add tools and use them.fromllmfuncs.toolimportTool,ToolCollectiontool_collection=ToolCollection()tool_collection.add_tool(tool)print(len(tool_collection))# 1You can also add tools in bulk from a module, a package, or a glob pattern.tool_collection.add_tools_from_module('some_module')tool_collection.add_tools_from_package('some_package')tool_collection.add_tools_from_glob('*.py')And here's how to use a tool:json_args='{"name": "World"}'result=tool_collection.use_tool("greet",json_args)print(result)# "Hello, World!"For more detailed usage and examples, please check the API documentation and the example scripts in theexamplesfolder.Creating New ToolsIf you have an existing module that you want to work withllmfuncs, you can try passing it through an LLM
to generate the type hints and docstrings for any functions missing them. For example:Please update this module to add type hints to all function parameters and Google style docstrings to each function.ContributeWe welcome contributions tollmfuncs!LicenseThis project is licensed under the terms of the MIT license. For more details, see theLICENSEfile.
Package: llm-functions
LLM Functions ⚙️Easily define custom LLM (Language Learning Model) functions with structured input and output.Getting Started1. InstallationInstall the package using pip:pipinstallllm-functions2. Setting Up Your First FunctionLet's create a sentiment classification function:Step 1: Create a directory named sentiment. Inside this directory, you need two files:└─sentiment├──function_args.json├──model_args.json└──template.txtStep 2:Define your prompt template in template.txt:Aalyzeanddeterminewhetherthesentimentofthefollowingsentenceispositive,negative,orneutral.
Sentence:{sentence}Step 3:Set model paramters in model_args.json:{"temperature":0.0,"model":"gpt-3.5-turbo"}Step 4:Specify the function's output format in function_args.json:{"function_name":"sentiment_classifier","description":"Classify the sentiment of a sentence","properties":{"thoughts":{"type":"string","description":"Your thoughts when classifying sentiment of the given sentence."},"sentiment":{"type":"string","enum":["positive","negative","neutral"],"description":"The sentiment of the given sentence."}}}3. Using Your LLM Function in PythonHere's how you can load and use your LLM function:fromllm_functionsimportLLMFunction# Load your functionsentiment=LLMFunction.from_dir("./sentiment/")# Make a predictionpred=sentiment(sentence="I am super happy!")# Expected Output:# {#   "rationale": "The sentiment in the sentence is positive",#   "sentiment": "positive"# }
Package: llm-gemini
llm-geminiAPI access to Google's Gemini modelsInstallationInstall this plugin in the same environment asLLM.llminstallllm-geminiUsageConfigure the model by setting a key called "gemini" to your API key:llmkeyssetgemini<paste key here>Now run the model using-m gemini-pro, for example:llm-mgemini-pro"A joke about a pelican and a walrus"Why did the pelican get mad at the walrus?Because he called him a hippo-crit.To chat interactively with the model, runllm chat:llmchat-mgemini-proDevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-gemini
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:llminstall-e'.[test]'To run the tests:pytest
Package: llm-groq
llm-groqLLMplugin providing access toGroqcloudmodels.InstallationInstall this plugin in the same environment as LLM:llminstallllm-groqUsageFirst, obtain an API key forGroqcloud.Configure the key using thellm keys set groqcommand:llmkeyssetgroq<paste key here>You can now access the three Mistral hosted models:groq-llama2andgroq-mixtral.To run a prompt throughgroq-mixtral:llm-mgroq-mixtral'A sassy name for a pet sasquatch'To start an interactive chat session withgroq-mixtral:llmchat-mgroq-mixtralllm chat -m groq-mixtral
Chatting with groq-mixtral
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> three proud names for a pet walrus
Here are three whimsical and proud-sounding names for a pet walrus:

1. Regalus Maximus
2. Glacierus Royalty
3. Arctican Aristocat

These names evoke a sense of majesty and grandeur, fitting for a noble and intelligent creature like a walrus. I hope you find these names fitting and amusing! If you have any other requests or need assistance with something else, please don't hesitate to ask.To use a system prompt withgroq-mixtralto explain some code:catexample.py|llm-mgroq-mixtral-s'explain this code'Model optionsTBDDevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-groq
python3-mvenvvenvsourcevenv/bin/activate
Package: llm-guard
LLM Guard - The Security Toolkit for LLM InteractionsLLM Guard byProtect AIis a comprehensive tool designed to fortify the security of Large Language Models (LLMs).Documentation|Playground|ChangelogWhat is LLM Guard?By offering sanitization, detection of harmful language, prevention of data leakage, and resistance against prompt
injection attacks, LLM-Guard ensures that your interactions with LLMs remain safe and secure.InstallationBegin your journey with LLM Guard by downloading the package:pipinstallllm-guardGetting StartedImportant Notes:LLM Guard is designed for easy integration and deployment in production environments. While it's ready to use
out-of-the-box, please be informed that we're constantly improving and updating the repository.Base functionality requires a limited number of libraries. As you explore more advanced features, necessary libraries
will be automatically installed.Ensure you're using Python version 3.9 or higher. Confirm with:python --version.Library installation issues? Consider upgrading pip:python -m pip install --upgrade pip.Examples:Get started withChatGPT and LLM Guard.Deploy LLM Guard asAPISupported scannersPrompt scannersAnonymizeBanCompetitorsBanSubstringsBanTopicsCodeInvisibleTextLanguagePromptInjectionRegexSecretsSentimentTokenLimitToxicityOutput scannersBanCompetitorsBanSubstringsBanTopicsBiasCodeDeanonymizeJSONLanguageLanguageSameMaliciousURLsNoRefusalReadingTimeFactualConsistencyRegexRelevanceSensitiveSentimentToxicityURLReachabilityRoadmapYou can find our roadmaphere. Please don't hesitate to contribute or create issues, it helps us improve LLM Guard!Community, Contributing, Docs & SupportLLM Guard is an open source solution.
We are committed to a transparent development process and highly appreciate any contributions.
Whether you are helping us fix bugs, propose new features, improve our documentation or spread the word,
we would love to have you as part of our community.Give us a ⭐️ github star ⭐️ on the top of this page to support what we're doing,
it means a lot for open source projects!Read ourdocsfor more info about how to use and customize LLM Guard, and for step-by-step tutorials.Post aGithub
Issueto submit a bug report, feature request, or suggest an improvement.To contribute to the package, check out ourcontribution guidelines, and open a PR.Join our Slack to give us feedback, connect with the maintainers and fellow users, ask questions,
get help for package usage or contributions, or engage in discussions about LLM security!Production SupportWe're eager to provide personalized assistance when deploying your LLM Guard to a production environment.Send Email ✉️
Package: llm-horde
llm-hordeLLMplugin for models hosted byAI HordeInstallationFirst,install the LLM command-line utility.Now install this plugin in the same environment as LLM.llminstallhttps://github.com/spinagon/llm-horde/archive/refs/heads/main.zipConfigurationFirstRegister an accountwhich will generate for you an API key. Store that key somewhere.If you do not want to register, you can use '0000000000' as api_key to connect anonymously. If you don't set your key, it will be used automatically. However anonymous accounts have the lowest priority when there's too many concurrent requests!To increase your priority you will need a unique API key and then to increase your Kudos.Read how Kudos are working.You can set that as an environment variable calledAIHORDE_KEY, or add it to thellmset of saved keys using:llmkeyssetaihordeEnter key: <paste key here>UsageTo list available models, run:llmmodelslistYou should see a list that looks something like this:AI Horde: horde/Henk717/airochronos-33B
AI Horde: horde/tgi-fp16-8k/Xwin-LM/Xwin-LM-70B-V0.1
AI Horde: horde/Gryphe/MythoMax-L2-13bTo run a prompt against a model, pass its full model ID to the-moption, like this:llm-mhorde/Henk717/airochronos-33B"Five spooky names for a pet vampire bat"You can set a shorter alias for a model using thellm aliasescommand like so:llmaliasessetairochronoshorde/Henk717/airochronos-33BNow you can prompt it using:catllm_horde.py|llm-mairochronos-s'write some pytest tests for this'If you use "horde" as a model name, you can select one or more models matching a regex pattern using "pattern" option:llm-mhorde-opattern"13B""What is the capital of Lebanon?"You can change the maximum length of returned text by using "max_tokens" option (80 by default), most workers will not :llm-mhorde-opatternmythomax-s'Write a sonnet'-omax_tokens120If you want to continue generating text without new prompt, try:llm-c""To start interactive chat, typellmchat-mhorde-opatternllama2
Package: llm-http-api
llm-http-apiHTTP API forLLMwith OpenAI compatibilityUsage>llmhttp-api--help
Usage:llmhttp-api[OPTIONS]RunaFastAPIHTTPserverwithOpenAIcompatibility

Options:-h,--hostTEXT[default:0.0.0.0]-p,--portINTEGER[default:8080]-l,--log-levelTEXT[default:info]--helpShowthismessageandexit.>curlhttp://localhost:8080/v1/embeddings-XPOST-H"Content-Type: application/json"-d'{"input": "Hello world","model": "jina-embeddings-v2-small-en"}'{"object":"embedding","embedding":[-0.47561466693878174,-0.4471365511417389,...],"index":0}Supported OpenAI EndpointsModelsGET /v1/modelsGET /v1/models/{model}DELETE /v1/models/{model}EmbeddingsPOST /v1/embeddingsUnsupported OpenAI EndpointsA detailed list of unimplemented OpenAI endpoints can be foundhereDevelopmentThis repository manages the dev environment as a Nix flake and requiresNix to be installednixdevelop-c$SHELLmakedeps.install
makedeps.install/testmaketestmakecoveragemakelintmakeformat
Package: llm-inference
Large Language Model (LLM) Inference API and Chatbot 🦙Inference API for LLMs like LLaMA and Falcon powered by Lit-GPT fromLightning AIpip install llm-inferenceInstall from main branchpipinstallgit+https://github.com/aniketmaurya/llm-inference.git@mainNote: You need to manually installLit-GPTand setup the model weights to use this project.pip install lit_gpt@git+https://github.com/aniketmaurya/install-lit-gpt.git@installFor Inferencefromllm_inferenceimportLLMInference,prepare_weightsfromrichimportprintpath=prepare_weights("EleutherAI/pythia-70m")model=LLMInference(checkpoint_dir=path)print(model("New York is located in"))How to use the Chatbotfromllm_chainimportLitGPTConversationChain,LitGPTLLMfromllm_inferenceimportprepare_weightsfromrichimportprintpath=str(prepare_weights("lmsys/longchat-13b-16k"))llm=LitGPTLLM(checkpoint_dir=path,quantize="bnb.nf4")# 8.4GB GPU memorybot=LitGPTConversationChain.from_llm(llm=llm,verbose=True)print(bot.send("hi, what is the capital of France?"))Launch Chatbot App1. Download weightsfromllm_inferenceimportprepare_weightspath=prepare_weights("lmsys/longchat-13b-16k")2. Launch Gradio Apppython examples/chatbot/gradio_demo.pyFor deploying as a REST APICreate a Python fileapp.pyand initialize theServeLLaMAApp.# app.pyfromllm_inference.serveimportServeLLaMA,Response,PromptRequestimportlightningasLcomponent=ServeLLaMA(input_type=PromptRequest,output_type=Response)app=L.LightningApp(component)lightningrunappapp.py
Package: llm-inflation
LLM-InflationCode for my Machine Learning for Trading and Portfolio Management course project at ENSAEInstallationThe package was published to PyPi, and can be installed usingpipinstallllm-inflationUsagePlease refer to the Colab notebook, and the code's docstrings.Code standardsThis project adheres to the defaultblackandflake8standards, enforced with pre-commit hooks.
Package: llmio
llmioEasily connect large language models into your application.Setuppip install llmioExampleimportllmioassistant=llmio.Assistant(description="""You are a calculator.Always use the provided commands to perform calculations,never try to calculate on your own.When being given a math problem, do not explain the steps,only execute them necessary commands and then present the answer.""",key="<openai-key>",)@assistant.commanddefadd(num1:float,num2:float)->float:"""Add two numbers"""print(f"Adding{num1}+{num2}")returnnum1+num2@assistant.commanddefmultiply(num1:float,num2:float)->float:"""Multiply two numbers"""print(f"Multiplying{num1}*{num2}")returnnum1*num2reply,_=assistant.speak("calculate the answer of (10 + 20) * 1337")print(reply)
Package: llm-json-adapter
LLM JSON AdapterWhat is it ?When using LLMs from the system, you often expect to get output results in JSON: OpenAPI's GPT API has a mechanism called Function Calling, which can return JSON, but Google's Gemini does not seem to have that functionality.Therefore, I have created a wrapper library to switch LLMs and get results in JSON. What this library can do is as follows.Allows you to define the results you want to get in JSON SchemaSwitch between LLMs (currently supports OpenAI's GPT and Google's Gemini).Retry a specified number of times if the JSON retrieval failsHow to useUse the following code to get the results in JSON.ParameterDescriptionprovider_nameThe name of the LLM provider to use. Currently, only "google" and "openai" are supported.max_retry_countThe number of times to retry if the JSON retrieval fails.attributesThe attributes to pass to the LLM provider.AttributesParameterDescriptionapi_keyThe API key to use.fromllm_json_adapterimportLLMJsonAdapter,Responseadapter=LLMJsonAdapter(provider_name="google",max_retry_count=3,attributes={"api_key":"Your API Key"})result=adapter.generate(prompt="prompt",language="en",act_as="Professional Software Service Business Analyst",function=Response(name="response name",description="response description",parameters={"type":"object","properties":{"data":{"type":"array","items":{"type":"object","properties":{"title":{"type":"string",},"description":{"type":"string",},},"required":["title","description"],},},},"required":["data"]},))