Saravanan666/pypi-org huggingface dataset:
Package: llm-commons
LLM ToolsThis package wraps the OpenAI API and other LLM models that share the completion endpoint feature. It provides a simple interface to generate text from a prompt.Installationpipinstallllm-toolsUsagefromllm_tools.openaiimportpromptfrompathlibimportPath# Create a prompt object that will be logged to the `my-task` directory under the# current working directory.# The PromptZeros class will use arguments that will maximize the determinism of the outputs.pz=prompt.PromptZeros(Path("./"),"my-task")# Send a prompt to the API and return the response.w=pz.send_prompt("What is the WDI code for GDP?",return_data=True)# Print the response.print(w)# Output:# {'message': [{'role': 'system', 'content': ''}, {'role': 'user', 'content': 'What is the WDI code for GDP?'}], 'response': {'id': 'chatcmpl-8V0IN0VbeYTrsBAohs309DHJQLJL8', 'choices': [{'finish_reason': 'stop', 'index': 0, 'message': {'content': 'The WDI code for GDP (Gross Domestic Product) is "NY.GDP.MKTP.CD".', 'role': 'assistant', 'function_call': None, 'tool_calls': None}}], 'created': 1702399995, 'model': 'gpt-3.5-turbo-0613', 'object': 'chat.completion', 'system_fingerprint': None, 'usage': {'completion_tokens': 23, 'prompt_tokens': 20, 'total_tokens': 43}}, 'content': 'The WDI code for GDP (Gross Domestic Product) is "NY.GDP.MKTP.CD".', 'api_kwargs': {'temperature': 0, 'top_p': 0, 'n': 1, 'stream': False, 'frequency_penalty': 0, 'presence_penalty': 0, 'model': 'gpt-3.5-turbo', 'max_tokens': 4029}, 'metadata': {}, 'message_hash': '1a5816095c232f29428d4cec70498391'}NOTE: The output is also logged to themy-taskdirectory under the current working directory.
Package: llmdataset
LLM datasetHow to useinstallhttps://pypi.org/project/llmdataset/pip install llmdatasetsettingdataloader=llmdataset.dataloader(data_type='train',batch_size=4,seed=3655,max_data=5)executionfor i in dataloader:
    print('------')
    print(i)Useful datasetGSM8Khttps://huggingface.co/datasets/gsm8kllmdataset=main.LLMdataset(dataset_name="gsm8k",subset='main')MultiArithhttps://huggingface.co/datasets/ChilleD/MultiArithllmdataset=main.LLMdataset(dataset_name="ChilleD/MultiArith")Big Benchhttps://huggingface.co/datasets/bigbenchllmdataset=main.LLMdataset(dataset_name="tasksource/bigbench",subset="abstract_narrative_understanding")Big Bench Hardhttps://huggingface.co/datasets/lukaemon/bbhllmdataset=main.LLMdataset(dataset_name="reasoning-machines/gsm-hard",subset="boolean_expressions")GSM Hardhttps://huggingface.co/datasets/reasoning-machines/gsm-hardllmdataset=main.LLMdataset(dataset_name="reasoning-machines/gsm-hard")wikitablequestionshttps://huggingface.co/datasets/wikitablequestions?row=0llmdataset=main.LLMdataset(dataset_name="wikitablequestions")StrategyQAhttps://huggingface.co/datasets/ChilleD/StrategyQA?row=0llmdataset=main.LLMdataset(dataset_name="ChilleD/StrategyQA")ARChttps://huggingface.co/datasets/allenai/ai2_arcllmdataset=main.LLMdataset(dataset_name="allenai/ai2_arc",subset="ARC-Challenge")AQuA-RAThttps://huggingface.co/datasets/aqua_rat/viewer/rawllmdataset=main.LLMdataset(dataset_name="aqua_rat",subset="raw")GPQAhttps://huggingface.co/datasets/Idavidrein/gpqallmdataset=main.LLMdataset(dataset_name="Idavidrein/gpqa",subset="gpqa_diamond")SVAPhttps://huggingface.co/datasets/ChilleD/SVAMP?row=0llmdataset=main.LLMdataset(dataset_name="ChilleD/SVAMP")CommonsenseQAhttps://huggingface.co/datasets/tau/commonsense_qallmdataset=main.LLMdataset(dataset_name="tau/commonsense_qa")
Package: llmdeploy
OpenMMLab websiteHOTOpenMMLab platformTRY IT OUTEnglish |简体中文IntroductionInstallationBelow are quick steps for installation:condacreate-nopen-mmlabpython=3.8
condaactivateopen-mmlab
gitclonehttps://github.com/open-mmlab/llmdeploy.gitcdllmdeploy
pipinstall-e.Quick StartBuildPull docker imageopenmmlab/llmdeploy:baseand build llmdeploy libs in its launched containermkdirbuild&&cdbuild
../generate.sh
make-j$(nproc)&&makeinstallServingLLaMAWeights for the LLaMA models can be obtained from by filling outthis formRun one of the following commands to serve a LLaMA model on NVIDIA GPU server:7Bpython3llmdeploy/serve/fastertransformer/deploy.pyllama-7B/path/to/llama-7bllama\--tokenizer_path/path/to/tokenizer/model
bashworkspace/service_docker_up.sh--lib-dir$(pwd)/build/install/backends/fastertransformer13Bpython3llmdeploy/serve/fastertransformer/deploy.pyllama-13B/path/to/llama-13bllama\--tokenizer_path/path/to/tokenizer/model--tp2bashworkspace/service_docker_up.sh--lib-dir$(pwd)/build/install/backends/fastertransformer33Bpython3llmdeploy/serve/fastertransformer/deploy.pyllama-33B/path/to/llama-33bllama\--tokenizer_path/path/to/tokenizer/model--tp4bashworkspace/service_docker_up.sh--lib-dir$(pwd)/build/install/backends/fastertransformer65Bpython3llmdeploy/serve/fastertransformer/deploy.pyllama-65B/path/to/llama-65bllama\--tokenizer_path/path/to/tokenizer/model--tp8bashworkspace/service_docker_up.sh--lib-dir$(pwd)/build/install/backends/fastertransformerServingVicuna7Bpython3-mpipinstallfschat
python3-mfastchat.model.apply_delta\--base-model-path/path/to/llama-7b\--target-model-path/path/to/vicuna-7b\--delta-pathlmsys/vicuna-7b-delta-v1.1

python3llmdeploy/serve/fastertransformer/deploy.pyvicuna-7B/path/to/vicuna-7bhf
bashworkspace/service_docker_up.sh--lib-dir$(pwd)/build/install/backends/fastertransformer13Bpython3-mpipinstallfschat
python3-mfastchat.model.apply_delta\--base-model-path/path/to/llama-13b\--target-model-path/path/to/vicuna-13b\--delta-pathlmsys/vicuna-13b-delta-v1.1

python3llmdeploy/serve/fastertransformer/deploy.pyvicuna-13B/path/to/vicuna-13bhf
bashworkspace/service_docker_up.sh--lib-dir$(pwd)/build/install/backends/fastertransformerInference with Command Line Interfacepython3llmdeploy/serve/client.py{server_ip_addresss}:333371Inference with Web UIpython3llmdeploy/webui/app.py{server_ip_addresss}:33337model_nameUser GuideQuantizationIn fp16 mode, kv_cache int8 quantization can be enabled, and a single card can serve more users.
First execute the quantization script, and the quantization parameters are stored in the weight directory transformed bydeploy.py.
Then adjustconfig.iniuse_context_fmhachanged to 0, means offquant_policyis set to 4. This parameter defaults to 0, which means it is not enabledContributingWe appreciate all contributions to LLMDeploy. Please refer toCONTRIBUTING.mdfor the contributing guideline.AcknowledgementFasterTransformerLicenseThis project is released under theApache 2.0 license.
Package: llmdet
🪬 LLMDet：A Large Language Models Detection ToolLLMDet is a text detection tool that can identify which generated sources the text came from (e.g. large language model or human-write). The core idea of the detection algorithm is to use the n-grams probability sampled from specified language model to calculate proxy perplexity of large language models, and use the proxy perplexity as a feature to train a text classifier.FeaturesWe believe that a practical LLM detection tool needs to have the following capabilities, which is also the goal of our LLMDet.Specificity: Our project aims to distinguish between different large-scale language models and human-generated text. For example, LLMDet can tell you whether the text is generated by GPT-2 or OPT or a human, and give each a specific probability.Safty: Our project does not need to require running large language models locally. That is, we can act as a third-party authentication agent without maintaining large language models, which may be fixed assets or sensitive information for large companies.Efficiency: Our method detects very fast. This is because we don't need to infer from large language models.Extendibility: Our project can easily adapt to newly proposed large language models.Installation NotesA package for large language model-generated text detection tool.Code is compatible with Python >=3.8Fully automatic installation:pip install llmdetSemi-automatic installation: First downloadhttp://pypi.python.org/pypi/llmdet/, decompress and runpython setup.py installSeerequirements.txtfor dependent python packages.Main FunctionsCurrently, it is supported to determine whether the text comes from GPT-2, OPT, UniLM or Human-write.Examplesimportllmdetllmdet.load_probability()text="The actress was honoured for her role in 'The Reader' at the annual ceremony, which was held at the Royal Albert Hall. The film, which is based on the novel by the same name by Philip Roth, tells the story of a New York Times reporter who returns to his hometown to cover the death of his brother-in-law. Winslet plays his wife, with whom he has been divided since the death of their son.\nIn the film, Winslet plays the mother of the grieving brother-in-law.\nThe actress also won a Golden Globe for her role in the film at the ceremony in November.\nWinslet was also nominated for an Oscar for her role in 'The Reader'.\nThe 63-year-old Winslet was seen accepting her awards at the ceremony, where she was joined by her husband, John Krasinski, who has been nominated for best supporting actor in the film.\nWinslet and Krasinski met while"# Detect, `text` is a string or string listresult=llmdet.detect(text)print(result)Detection Results[{'OPT':0.5451331013247862,'GPT-2':0.4393605735865629,'UniLM':0.012642800848279893,'T5':0.0022592730436008556,'Bloom':0.00025873253035729044,'GPT-neo':0.0002520776780109571,'LLaMA':6.0459794454546154e-05,'Human_write':1.9576671778802474e-05,'BART':1.3404522168622544e-05}]TODO ListExtend the detection models to LLaMA, T5, Bart and Vicuna.Compress the project.Add more experimental performnce results.Citation
Package: llm-director
LLM-DirectorIntroduction 📖A light-weight Python library 🐍 for deploying DAG chains of customizable Large Language Model operations. Leveraging the asyncio framework, LLM-Director allows developers to create, manage, and orchestrate actions and processes in an efficient and scalable manner. With its unique structure and integration with FastAPI, LLM-Director offes a robust solution for building applications, API endpoints, and workflow automation systems.Quick Install⚡To get started with LLM-Director, you can install the package using pip:pipinstallllm-directorEnsure you have Python 3.7 or higher installed, as LLM-Director relies on the latest features of asyncio and FastAPI.What You Can Build With LLM-Director 🏗️With LLM-Director, you can create a variety of applications and systems:Asynchronous Workflow Systems: Design and execute complex workflows with asynchronous actions, enhancing the efficiency and responsiveness of your applications.API Endpoints: Easily integrate with FastAPI to expose your workflows as RESTful API endpoints, allowing for seamless interaction with front-end applications or external systems.Automated Data Processing Pipelines: Implement pipelines for data processing, where each step can be a distinct action, conditionally processed, and managed by the Director.Event-Driven Architectures: Build systems that react to specific events, trigger actions, and handle a series of responses in an orderly and scalable manner.Documentation 📚The LLM-Director library consists of several key components:Director: The central orchestrator that manages actions and listeners.Action: The basic unit of functionality, representing a single task or process.Blocks: Specialized actions such as Split, Condition, Termination, Merge, and Save, each serving a unique purpose in the workflow.FastAPI Integration: Methods to expose actions and workflows as API endpoints.For detailed documentation, visit LLM-Director Documentation (Coming Soon!).Contributing 🤝Please Contribute! I hope to grow this fast, as I think many people will benefit from this framework.
Package: llmem
llmemA read through cache for OpenAI chat completions.Why:Avoid running up bills when developing/testing by sending the same request multiple times/forgetting to mock things.Fine-tune an open source model on data I've paid forAdhere to all terms of use.Evaluate future models.Track model changes over time.useIt currently only supports the chat completion API.There is one additional (optional) field in the request,age. You can give it a number + characters like10wit will only return from cache if the entry is younger than that (in this case 10 weeks). Acceptable characters are s(econds), m(inute), h(hours), d(ays), w(eeks), y(ears).requestsNOTE: These examples run against my public serverllmem.com. Youmight want to run your own server instead.importrequestsheaders={"Content-Type":"application/json","Authorization":f"Bearer{api_key}"}payload={"model":"gpt-3.5-turbo","messages":[{"role":"user","content":[{"type":"text","text":"Hi."},]}],"max_tokens":10}response=requests.post("https://llmem.com/chat/completions",headers=headers,json=payload)print(response.json())If you run that twice, you'll see that first it gives an ID starting withchatcmpl, then IDs starting withllmem.You can also use the OpenAI client, although it will currently only work for/chatrequests:fromopenaiimportOpenAIclient=OpenAI(api_key=api_key,base_url="https://llmem.com")chat_completion=client.chat.completions.create(messages=[{"role":"user","content":"Say this is a test",}],model="gpt-3.5-turbo",)private serverdockerMakes it available on port8000, and keeps its cache in~/.llmem.dockerrun--rm-p8000:8000-v~/.llmem:/workspaceghcr.io/c0g/llmem:latestpythonpipinstallllmem
uvicornllmem:appBy default it will make allmem.dbfile in the working directory, you can override this location by setting theLLMEM_DB_FILEenvironment variable.You can get the codeon GitHub.public serverI host a public server atllmem.com. If you point your API client tohttps://llmem.com/chat, you'll hit my cache and share the wealth (and get to share other people's wealth).NOTE: I could look at your OpenAI key if I wanted to. I don't, but I could. You can probably trust me[^criminal].NOTE: I can look at your queries. I probably will (see 'Why' bullets above). Please don't submit anything you wouldn't want your mother seeing, since I might well be your mother.NOTE: If any of the aboveNOTEs worry you it's probably best not to point your API client tohttps://llmem.com.[^criminal]: That's exactly what a criminal would say.
Package: llm-embed-hazo
llm-embed-hazoDummyllmmodel for demonstration purposesInstallationInstall this plugin in the same environment asLLM.llm install llm-embed-hazoUsageThe models will be downloaded the first time you try to use them.Seethe LLM documentationfor everything you can do.To get started embedding a single string, run the following:DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-embed-hazo
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:llminstall-e'.[test]'To run the tests:pytest
Package: llm-embed-jina
llm-embed-jinaEmbedding models from Jina AIBackgroundJina AI Launches World's First Open-Source 8K Text Embedding, Rivaling OpenAIintroduces these models.See alsoEmbeddings: What they are and why they matterfor background on embeddings and an explanation of the LLM embeddings tool.InstallationInstall this plugin in the same environment asLLM.llm install llm-embed-jinaUsageThis plugin adds support for three new embedding models:jina-embeddings-v2-small-en: 33 million parameters.jina-embeddings-v2-base-en: 137 million parameters.jina-embeddings-v2-large-en: 435 million parameters - not yet released, but it will work once it has been released.The models will be downloaded the first time you try to use them.Seethe LLM documentationfor everything you can do.To get started embedding a single string, run the following:llmembed-mjina-embeddings-v2-small-en-c'Hello world'This will output a JSON array of 512 floating point numbers to your terminal.DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-embed-jina
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:llminstall-e'.[test]'To run the tests:pytest
Package: llm-embed-onnx
llm-embed-onnxRun embedding models using ONNXThis LLM plugin is a wrapper aroundonnx_embedding_modelsby Benjamin Anderson.InstallationInstall this plugin in the same environment asLLM.llminstallllm-embed-onnxUsageThis plugin adds the following embedding models, which can be listed usingllm embed-models:onnx-bge-micro
onnx-gte-tiny
onnx-minilm-l6
onnx-minilm-l12
onnx-bge-small
onnx-bge-base
onnx-bge-largeYou can run any of these models usingllm embedcommand:llmembed-monnx-bge-micro-c"Example content"This will output a 384 length JSON array of floating point numbers, starting:[-0.03910085942622519, -0.0030843335461659795, 0.032797761260860724,The first time you use any of these models the model will be downloaded to thellm_embed_onnxdirectory in yourLLM data directory. On macOS this defaults to:~/Library/Application Support/io.datasette.llm/llm_embed_onnxFor more on how to use these embedding models seethe LLM embeddings documentation.DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-embed-onnx
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:llminstall-e'.[test]'To run the tests:pytest
Package: llments
llmentsDevelopment InformationTo start developing, install the development dependencies and pre-commit hooks:pipinstall".[dev]"pre-commitinstallThere are severalpre-commit hooksthat will run on every
commit to perform formatting, typing, and linting.ruff- Runs formatting, import sorting, and linting.mypy- Runs type checking.markdownlint- Runs markdown linting.yamllint- Runs YAML linting.
Package: llmeval
llmevalLLM evaluation and test framework to build internal test harnesses and automate reliable deployment workflowsPrerequisite Softwarepdm-pip install pdmDevelopmentRunpdm syncto install all dependencies managed by pdm tool.If you need to add new dependency, runpdm add [dependency-name].Runpython llmeval/main.pyfor the cli.Runpython llmeval/main.py eval hydra.mode=MULTIRUNfor evaluation.Format filesRunblack llmevalto format all the py files.Build & PublishRunpdm buildRunpdm publish
Package: llm-explorer
Occlusion LLM ExplorerLakehouse Analytics & Advanced MLSetupImportantThis package requiresOpen AI & HuggingFace API key. Remember to run from a folder with the.streamlit/secrets.tomlfile.
Seehereandherefor more details.Quick Installpython-mpipinstallllm-explorerllm_explorerInitial load could take some time as it downloads the model and the tokenizer. Remember to include the secrets.toml file under .streamlit/ folder.Build from sourceClone the repositorygitclonehttps://github.com/Occlusion-Solutions/llm_explorer.gitInstall the packagecdllm_explorer&&makeinstallRun the packagellm_explorerBuild manauallyAfter cloning, ceate a virtual environmentcondacreate-nllm_explorerpython=3.10
condaactivatellm_explorerInstall the requirementspipinstall-rrequirements.txtRun the python installationpythonsetup.pyinstall
llm_explorerUsageUse thedemo@occlusion.solutionsuser andDEMO@occlusionpassword to login.The deployment requires a secrets.toml file created under .streamlit/:touch.streamlit/secrets.tomlIt should have a schema like this:[connections.openai]api_key="sk-..."# OpenAI API Key[connections.huggingface]api_key="shf_..."# HuggingFace API Key[connections.databricks]server_hostname="your databricks host"http_path="http path under cluster JDBC/ODBC connectivity"access_token="your databricks access token"Run ModesChainAn assistant Query engine, that is asked naturally with table references and helps in the query generation. The execution of the queries is manualAgentIt uses the pandas agent to generate the queries and execute them. It is a more natural way of querying the data and it operates autonomously until it thinks it finds and answer.ChatIt uses the HuggingFace Transformers Agent chat to operate in a conversational way.Lakehouse Agent SampleAgent is queried for the top 10 producing wells. It identifies the tables it has access to and understands that the request could be satified by the padalloc table. It then creates a query that returns the top 10 producing assets and return the results.>EnteringnewAgentExecutorchain...

Observation:logs,wells
Thought:Ishouldlookattheschemaofthemicrochip_logsandpadalloctablestoseewhatcolumnsIcanuse.

Action:schema_sql_db
ActionInput:"wells"Observation:DDL
Thought:Ishouldquerythepadalloctabletogetthetop10producingwells.

Action:query_sql_db
ActionInput:"SELECT WELL_CODE, SUM(PROD_GAS_VOLUME_MCF) AS total_gas_volume_mcf FROM padalloc GROUP BY WELL_CODE ORDER BY total_gas_volume_mcf DESC LIMIT 10"Observation:results_dataframe
Thought:Inowknowthetop10producingwells.

FinalAnswer:Thetop10producingwellsare1222344,1212560,1222345,1212503,1222335,1222340,1222338,1222367,1220189,and1222352.

>Finishedchain.AttributionThis is an adapted implementation from the GitHub repository. See the contibutions list for more details:https://github.com/kaarthik108/snowChat
Package: llm-falcon-model
LLM Falcon modelllm_falcon_modelallows you to run a part of a Falcon model as a standalone PyTorch module.
This enables you to run in distributed mode, using even old GPUs with less memory.It only contains code needed for inference.
The only dependencies aretorch,tokenizers, andllm_sepweight.The original implementation is availablehere.Use it when you cannot fit the whole Falcon model into memory. If you have multiple
old GPUs with less memory, you can run different parts of the Falcon model on each of them and when
you make them communicate (using for examplesocket_rpc), you can run the full model on multiple
heterogeneous hosts. For example, if you have 4 old gaming PCs with a 3090 card (~6000$), you can run Falcon 40B
real-time (5-6 tokens/s)You can also use it when you want to run Falcon on a large number of inputs and have insufficient memory for the model.
You can serialize the intermediary results for all inputs and then continue with the next layersInstall with:pipinstallllm_falcon_modelOverviewThe most important methods of this microlib are:llm_falcon_model.load_tokenizer()- which loads an instance of theTokenizerfor the modelsllm_falcon_model.init_part(model_name, spec, device)- which creates a part of a Falcon model, by a given name (7b,40bor180b),
part specification (which layers you want to load, seesepweight part spec)
and a PyTorch device.llm_falcon_model.generate- which allows you to generate text based on a prompt.llm_falcon_model.score_batch- which allows you to score a bunch of possible continuations based on a prompt.llm_falcon_model.run_part- which allows you to run a part of Falcon in a distributed mode usingsocket_rpcQuick exampleimporttorchimportllm_falcon_modeltokenizer=llm_falcon_model.load_tokenizer()separated_weights_path='<PATH TO SEPARATED WEIGHTS>'model=llm_falcon_model.init_part(model_name='40b',spec='b 0-12',# Load begin and layers 0 to 12device='cuda:0')input_text="The world chess champion Magnus Carlsen"input_ids=tokenizer.encode(input_text).idsbatch=torch.tensor(input_ids).unsqueeze(0)x=model(batch)# x is now the result after end layer 12, shaped:# torch.Size([1, 7, 8192])