Saravanan666/pypi-org huggingface dataset:
Package: gpswe
gpswegpswe- this module is a Python client library for work with GPS Trackers (Wialon IPS, EGTS)InstallationInstall the current version withPyPI:pipinstallgpsweUsageFirst fill in the module Settings with your datafromgpswe.config.settingsimportsettingsasgpswe_settings#The port on which the server will be startedgpswe_settings.PORT=10500#The host on which the server will be startedgpswe_settings.HOST="0.0.0.0"#Supported version of WialonIPSgpswe_settings.WIALON_PROTOCOL_VERSION="2.0"#Host where the database is deployedgpswe_settings.POSTGRES_HOST="0.0.0.0"#The port on which the database is deployedgpswe_settings.POSTGRES_PORT=5432#The name of the database itselfgpswe_settings.POSTGRES_DB="gpswe"#The user's name to connect to the databasegpswe_settings.POSTGRES_USER="test_user"#The user's password to connect to the databasegpswe_settings.POSTGRES_PASSWORD="123"Next, you need to "initialize" your database (create all the necessary libraries in it)fromgpswe.gps_serverimportinit_dbinit_db()OK, now you can start the serverfromgpswe.gps_serverimportstart_serverstart_server()To get extended information about your data, you need to use the methodget_coordinates (imei,
from_datetime, to_datetime), where"imei"is the id of your tracker from which the data comes,"from_datetime"and"to_datetime"are the time interval in which you want to receive the dataimportdatetimefromgpswe.gps_serverimportget_coordinatesdata=get_coordinates(imei="1231233123",from_datetime=datetime.datetime(2022,5,15),to_datetime=datetime.datetime(2024,5,25))The data comes in json format{"speed":20.0,"distance":248.4,"coordinates":[{"latitude":61.2296333,"longitude":50.8259626},{"latitude":63.4578954564,"longitude":50.87545689874458}]}speed - average speed over a period of timedistance - distance traveledcoordinates - coordinates with latitude and longitudeAll messages will be written to the console via the standard logging python libraryContributingBug reports and/or pull requests are welcomeLicenseThe module is available as open source under the terms of theThe MIT License (MIT)
Package: gpt2
gpt2API client for GPT-2 text generator hosted on the cloud by Open Medical IOGenerate synthetic text from your custom prompt with  the latest released 774M model of OpenAI's GPT-2.
We take care of the GPU backend.
Seeopenmedical.io/gpt2for product demo and obtaining an API key!Installationpip install --upgrade gpt2Usagefromgpt2importClientc=Client('API SECRET')output_list=c.fast_generate(prompt='On a perfect summer day we went to')"""Generate text on the large 774M model using your own prompt and standard parameters.This only takes a few seconds per query.We're training additional models for your use in the near future, such as 774M finetuned on Chinese, Spanish, Wikipedia...We're also releasing finetuning APIs in September 2019.Stay tuned :)"""
Package: gpt2Api-Client
gpt2Api-ClientUnder construction!Developed by Anirudh from TechWithAnirudh (c) 2021Examples of How To Use (Alpha Version)Creating A Requeststart_paragraph="Today was a fine day. I woke up and brushed my teeth. And Ate Breakfast. I was going to work When"api=Api()api.setApiUrl(input('Enter The Api URL: '))print(api.ConditionalSamples(start_paragraph))start_paragraph="Today was a fine day. I woke up saw some news that the coding event is starting today. I love coding so I prepared. After I prepared"api=Api()api.setApiUrl(input('Enter The Api URL: '))print(api.ConditionalSamples(start_paragraph))Go to this url:https://github.com/techwithanirudh/Shared/blob/master/GPT2Api.ipynbAnd run the code
copy the url called GPT2ApiURL=...
and paste it here.
This will give the json responseVery Easy To To Use GPT2 WrapperInfoSee The Docs On How To Use This:https://techwithanirudh.blogspot.com/2021/03/docs-for-gpt2api-client-module.htmlCheck out:https://www.youtube.com/channel/UCmse86kaxJ3n1dDT_bXDukg
Package: gpt2-plot
A simple Python package that wraps existing model fine-tuning and generation scripts for OpenAI GPT-2 text generation model (specifically the "small", 124M hyperparameter version). Additionally, this package allows easier generation of text, generating to a file for easy curation, allowing for prefixes to force the text to start with a given phrase.UsageAn example for downloading the model to the local system, fineturning it on a dataset. and generating some text.Warning: the pretrained model, and thus any finetuned model, is 500 MB!importgpt_2_simpleasgpt2gpt2.download_gpt2()# model is saved into current directory under /models/124M/sess=gpt2.start_tf_sess()gpt2.finetune(sess,'shakespeare.txt',steps=1000)# steps is max number of training stepsgpt2.generate(sess)The generated model checkpoints are by default in/checkpoint/run1. If you want to load a model from that folder and generate text from it:importgpt_2_simpleasgpt2sess=gpt2.start_tf_sess()gpt2.load_gpt2(sess)gpt2.generate(sess)As with textgenrnn, you can generate and save text for later use (e.g. an API or a bot) by using thereturn_as_listparameter.single_text=gpt2.generate(sess,return_as_list=True)[0]print(single_text)You can pass arun_nameparameter tofinetuneandload_gpt2if you want to store/load multiple models in acheckpointfolder.NB:Restart the Python session firstif you want to finetune on another dataset or load another model.
Package: gpt2ppl-zh
本项目展示了使用中文 GPT2 预训练模型实现文档/语句的困惑度计算。pipinstallgpt2ppl-zh或者pipinstallgit+https://github.com/zejunwang1/gpt2ppl-zh句子困惑度计算示例可参考 gpt2ppl/demo_sent.py:# coding=utf-8importnumpyasnpfromgpt2pplimportGPT2PPLdefmain():sent1="机器学习与人工智能"sent2="机器学习与人工只能"model=GPT2PPL()"""
    model = GPT2PPL(
        model_name_or_path="uer/gpt2-chinese-cluecorpussmall",
        tokenizer_mode="bert",
        device="cuda",
        stride=512
    )
    """ppl1=model.get_ppl(sent1)ppl2=model.get_ppl(sent2)print("Sentence:{}, Perplexity Score:{}".format(sent1,ppl1))print("Sentence:{}, Perplexity Score:{}".format(sent2,ppl2))if__name__=="__main__":main()运行结果：Sentence: 机器学习与人工智能, Perplexity Score: 5.058856964111328
Sentence: 机器学习与人工只能, Perplexity Score: 74.32606506347656文档困惑度计算示例可参考
gpt2ppl/demo_doc.py，程序会计算文档中每个句子的困惑度：result=model.get_ppl_per_sentence(text,min_length=8)"""
result = model.get_ppl_per_sentence(
    text,
    ratio=0.4,
    min_length=8,
    max_length=510,
    return_loc=False
)
min_length: 分句的最小句子长度
max_length: 分句的最大句子长度
ratio: 句子中汉字字符的最小比例，若低于该阈值，则句子不参与困惑度计算
return_loc: 是否返回每个句子在原始文档中的字符位置
"""运行结果如下：The Perplexity Score of the whole text is:  5.978885650634766
The Perplexity Score for single sentences:
('中文分词是一种将连续的中文文本划分成有意义的词汇序列的自然语言处理技术。', 17.03954315185547)
('中文分词在中文文本处理中非常重要，因为汉字并没有空格或其他分隔符来区分单词，这给中文文本处理带来了困难。', 10.517306327819824)
('本文将介绍中文分词的原理和常用方法。', 19.19382667541504)
('中文分词的原理是基于词语在语料库中的频率来确定一个句子中的分词方案。', 18.930423736572266)
('分词的目标是将句子中的每个字符分配到一个词语中，从而获得有意义的词语序列。', 13.634822845458984)
('这样，文本就可以被更好地理解和分析。', 10.56257438659668)
('中文分词的方法有很多，其中最常用的是基于统计学的方法和基于规则的方法。', 8.299059867858887)
('基于统计学的方法通过对大量中文文本语料库的分析来得出一个概率模型，该模型根据词频和词语之间的关联性来判断每个字符的最佳分词位置。', 12.321337699890137)
('这种方法的优点是可以自适应地适应不同类型的文本，并且能够处理新词和生僻词，但是也有一些缺点，例如需要大量的语料库来训练模型，模型的效果受到数据质量和模型参数的影响。', 10.90810489654541)
('基于规则的方法则是依赖于一系列语法规则和词库，例如词典、词性标注、句法分析等，来判断每个字符的最佳分词位置。', 17.036846160888672)
('这种方法的优点是可以精确地控制分词结果，适合处理特定领域的文本，但是需要手动构建规则和词库，工作量比较大。', 14.91514778137207)
('除了基于统计学的方法和基于规则的方法，还有一些混合方法，例如将基于统计学的方法和基于规则的方法结合起来，以获得更好的分词效果。', 5.419482707977295)
('总之，中文分词是中文文本处理中的重要技术之一。', 15.306158065795898)
('在实际应用中，根据不同的需求和场景，可以选择不同的分词方法来获得最佳的分词结果。', 7.240055084228516)
Average Perplexity Score:  12.95176352773394
Burstiness Score:  4.184883485157172
Package: gpt-2-simple
A simple Python package that wraps existing model fine-tuning and generation scripts for OpenAI GPT-2 text generation model (specifically the "small", 124M hyperparameter version). Additionally, this package allows easier generation of text, generating to a file for easy curation, allowing for prefixes to force the text to start with a given phrase.UsageAn example for downloading the model to the local system, fineturning it on a dataset. and generating some text.Warning: the pretrained model, and thus any finetuned model, is 500 MB!importgpt_2_simpleasgpt2gpt2.download_gpt2()# model is saved into current directory under /models/124M/sess=gpt2.start_tf_sess()gpt2.finetune(sess,'shakespeare.txt',steps=1000)# steps is max number of training stepsgpt2.generate(sess)The generated model checkpoints are by default in/checkpoint/run1. If you want to load a model from that folder and generate text from it:importgpt_2_simpleasgpt2sess=gpt2.start_tf_sess()gpt2.load_gpt2(sess)gpt2.generate(sess)As with textgenrnn, you can generate and save text for later use (e.g. an API or a bot) by using thereturn_as_listparameter.single_text=gpt2.generate(sess,return_as_list=True)[0]print(single_text)You can pass arun_nameparameter tofinetuneandload_gpt2if you want to store/load multiple models in acheckpointfolder.NB:Restart the Python session firstif you want to finetune on another dataset or load another model.
Package: gpt3-simple-primer
gpt3-simple-primerSimple GPT-3 primer usingopenai.BackgroundGenerative Pre-trained Transformer 3 (GPT-3) is an autoregressive language model that uses deep learning to produce human-like text. For more information, visithttps://openai.com/blog/openai-api/.TheOpenAI Python libraryis the official Python wrapper for the OpenAI API. The purpose of this library is to simplify the priming process by providing easy to use methods for setting the instructions and adding examples.PrimingPriming is the practice of providing an initial prompt to the language model to improve subsequent model predictions.GPT-3 generally does very well even with short instructions and a few examples of your intended use case. Examples are typically delimited based on input and output. For instance, GPT-3 can be used to predict food ingredients based on the following prompt:Given the name of a food, list the ingredients used to make this meal.

Food: apple pie
Ingredients: apple, butter, flour, egg, cinnamon, crust, sugar

Food: guacamole
Ingredients: avocado, tomato, onion, lime, saltRequirementsYou will need an API key from OpenAI to access GPT-3.InstallationTo install, run:pip install gpt3-simple-primerUsageinput_textandoutput_textdetermines how input and output are delimited in the examples. The default is to useInputandOutput.from gpt3_simple_primer import GPT3Generator, set_api_key

KEY = 'sk-xxxxx'  # openai key
set_api_key(KEY)

generator = GPT3Generator(input_text='Food',
                          output_text='Ingredients')

generator.set_instructions('List the ingredients for this meal.')
generator.add_example('apple pie', 'apple, butter, flour, egg, cinnamon, crust, sugar')
generator.add_example('guacamole', 'avocado, tomato, onion, lime, salt')

# Ingredients: cream, egg yolk, sugar, lime, key lime juice
generator.generate(prompt='key lime pie',
                   engine='davinci',
                   max_tokens=20,
                   temperature=0.5,
                   top_p=1)To see the prompt used for priming:generator.get_prompt()To remove an example from the prompt:generator.remove_example('apple pie')ExamplesThe library includes examples of GPT-3 applications based off of specific prompts.from gpt3_simple_primer import set_api_key
from gpt3_simple_primer.examples import idiom_explainer

KEY = 'sk-xxxxx'  # openai key
set_api_key(KEY)

idiom_explainer.generate('hill to die on', max_tokens=15, engine='davinci')
Package: gpt3-tokenizer
AnOpenAIGPT3 helper library for encoding/decoding strings and counting tokens.Counting tokens gives the same output as OpenAI’stokenizerTested with versions:2.7.12,2.7.18and all3.x.xversionsInstallingpipinstallgpt3_tokenizerExamplesEncoding/decoding a stringimportgpt3_tokenizera_string="That's my beautiful and sweet string"encoded=gpt3_tokenizer.encode(a_string)# outputs [2504, 338, 616, 4950, 290, 6029, 4731]decoded=gpt3_tokenizer.decode(encoded)# outputs "That's my beautiful and sweet string"Counting tokensimportgpt3_tokenizera_string="That's my beautiful and sweet string"tokens_count=gpt3_tokenizer.count_tokens(a_string)# outputs 7
Package: gpt4all
Python GPT4AllThis package contains a set of Python bindings around thellmodelC-API.Package on PyPI:https://pypi.org/project/gpt4all/Documentationhttps://docs.gpt4all.io/gpt4all_python.htmlInstallationThe easiest way to install the Python bindings for GPT4All is to use pip:pip install gpt4allThis will download the latest version of thegpt4allpackage from PyPI.Local BuildAs an alternative to downloading via pip, you may build the Python bindings from source.PrerequisitesOn Windows and Linux, building GPT4All requires the complete Vulkan SDK. You may download it from here:https://vulkan.lunarg.com/sdk/homemacOS users do not need Vulkan, as GPT4All will use Metal instead.Building the python bindingsClone GPT4All and change directory:git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
cd gpt4all/gpt4all-backendBuild the backend.If you are using Windows and have Visual Studio installed:cmake -B build
cmake --build build --parallel --config RelWithDebInfoFor all other platforms:cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --parallelRelWithDebInfois a good default, but you can also useReleaseorDebugdepending on the situation.Install the Python package:cd ../../gpt4all-bindings/python
pip install -e .UsageTest it out! In a Python script or console:fromgpt4allimportGPT4Allmodel=GPT4All("orca-mini-3b-gguf2-q4_0.gguf")output=model.generate("The capital of France is ",max_tokens=3)print(output)GPU Usagefromgpt4allimportGPT4Allmodel=GPT4All("orca-mini-3b-gguf2-q4_0.gguf",device='gpu')# device='amd', device='intel'output=model.generate("The capital of France is ",max_tokens=3)print(output)Troubleshooting a Local BuildIf you're on Windows and have compiled with a MinGW toolchain, you might run into an error like:FileNotFoundError: Could not find module '<...>\gpt4all-bindings\python\gpt4all\llmodel_DO_NOT_MODIFY\build\libllmodel.dll'
(or one of its dependencies). Try using the full path with constructor syntax.The key phrase in this case is"or one of its dependencies". The Python interpreter you're using
probably doesn't see the MinGW runtime dependencies. At the moment, the following three are required:libgcc_s_seh-1.dll,libstdc++-6.dllandlibwinpthread-1.dll. You should copy them from MinGW
into a folder where Python will see them, preferably next tolibllmodel.dll.Note regarding the Microsoft toolchain: Compiling with MSVC is possible, but not the official way to
go about it at the moment. MSVC doesn't produce DLLs with alibprefix, which the bindings expect.
You'd have to amend that yourself.
Package: gpt4all-code-review
Code Review Automation ToolThis program is designed to assist developers by automating the process of code review. By leveraging a pre-trained standalone machine learning model (e.g., GPT-4), it reads source code files and provides suggestions for improvements.FeaturesFile Scanning:Ability to scan a single file or all files in the current directory.Multiple Output Formats:Results can be exported as plain text, JSON, or XML files.Customizable Model:Utilizes a customizable language model, allowing users to select the appropriate model for their needs.InstallationMake sure you have Python 3.x installed.pipinstall-rrequirements.txtorpip3install-rrequirements.txtUsageYou can run the program from the command line with various options:gpt4all_code_review--file<file_path>Optionsh,--help: Show this help message and exit--model: Specifies the model name. Default is"orca-mini-3b.ggmlv3.q4_0.bin".--file: Specifies the file path to analyze. If not provided, all files in the current directory will be analyzed.--all: Includes all files and folders in the current directory for scanning.--output: Output type (default:plain). Options:plain,txt(prettytable),json,xml.--export: Export to file (default:False).--export-folder: Export to folder (default:./code_review_results).ExampleTo analyze a single file:gpt4all_code_review--file=./path/to/yourfile.pyTo analyze all files in the current directory:gpt4all_code_review--allOutput FormatsPLAIN:A human-readable plain txt file.TXT:A human-readable table (prettytable) with file paths and suggestions.JSON:A machine-readable format that can be parsed programmatically.XML:An alternative machine-readable format.Dependenciesos: Standard Python library for interacting with the operating system.json: Standard Python library for working with JSON data.argparse: Standard Python library for parsing command-line arguments.gpt4all: A Python library for interfacing with GPT-4 models. Used to apply the AI models to the code.prettytable: A Python library to print tabular data in a visually appealing ASCII table format.datetime: Standard Python library for working with dates and times.console_progressbar: A Python library for displaying progress bars in the console.LicenseMIT License
Package: gpt4all-j
GPT4All-JPython bindings for theC++ portof GPT4All-J model.Please migrate toctransformerslibrary which supports more models and has more features.Installationpipinstallgpt4all-jDownload the model fromhere.Usagefromgpt4alljimportModelmodel=Model('/path/to/ggml-gpt4all-j.bin')print(model.generate('AI is going to'))Run in Google ColabIf you are gettingillegal instructionerror, try usinginstructions='avx'orinstructions='basic':model=Model('/path/to/ggml-gpt4all-j.bin',instructions='avx')If it is running slow, try building the C++ library from source.Learn moreParametersmodel.generate(prompt,seed=-1,n_threads=-1,n_predict=200,top_k=40,top_p=0.9,temp=0.9,repeat_penalty=1.0,repeat_last_n=64,n_batch=8,reset=True,callback=None)resetIfTrue, context will be reset. To keep the previous context, usereset=False.model.generate('Write code to sort numbers in Python.')model.generate('Rewrite the code in JavaScript.',reset=False)callbackIf a callback function is passed, it will be called once per each generated token. To stop generating more tokens, returnFalseinside the callback function.defcallback(token):print(token)model.generate('AI is going to',callback=callback)LangChainLangChainis a framework for developing applications powered by language models. A LangChain LLM object for the GPT4All-J model can be created using:fromgpt4allj.langchainimportGPT4AllJllm=GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin')print(llm('AI is going to'))If you are gettingillegal instructionerror, try usinginstructions='avx'orinstructions='basic':llm=GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin',instructions='avx')It can be used with other LangChain modules:fromlangchainimportPromptTemplate,LLMChaintemplate="""Question:{question}Answer:"""prompt=PromptTemplate(template=template,input_variables=['question'])llm_chain=LLMChain(prompt=prompt,llm=llm)print(llm_chain.run('What is AI?'))Parametersllm=GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin',seed=-1,n_threads=-1,n_predict=200,top_k=40,top_p=0.9,temp=0.9,repeat_penalty=1.0,repeat_last_n=64,n_batch=8,reset=True)C++ LibraryTo build the C++ library from source, please seegptj.cpp. Once you have built the shared libraries, you can use them as:fromgpt4alljimportModel,load_librarylib=load_library('/path/to/libgptj.so','/path/to/libggml.so')model=Model('/path/to/ggml-gpt4all-j.bin',lib=lib)LicenseMIT
Package: gpt4all-tone
GPT4ALL-Tone AnalyzerA comprehensive tool for sentiment analysis and tone review, empowered by GPT4ALL.InstallationUse the following command to installgpt4all-tone:pipinstallgpt4all-toneorpip3installgpt4all-toneUsageYou can use theToneAnalyzerclass to perform sentiment analysis on a given text. Here's a basic example of how you might use theToneAnalyzerclass:fromgpt4all_toneimportToneAnalyzer# Create an instance of the ToneAnalyzer classanalyzer=ToneAnalyzer("orca-mini-3b.ggmlv3.q4_0.bin","Wow it is great!")# Run the analyzerresult=analyzer.run()# Print the resultprint(result)# 1.0In this example, we're analyzing the text "Wow it is great!" and the result is1.0, indicating a positive sentiment.Command Line InterfaceYou can also use thegpt4all-tonecommand line interface to analyze text:gpt4all_tone"Wow it is great!"ContactAuthor:Evgenii EvstafevEmail:chigwel@gmail.comLicenseMIT License
Package: gpt4api-dg
DevGroundChatGPT API wrapperThis is a wrapper for theOpenAI's GPT-4 APIbyDevGround.ModelThe AI model used in this package is GPT-4LicenseThis package is licensed under the MIT license.LanguageThe language used in this package is Python.Requirementsrequires environment variables:OPENAI_API_KEYInstallationPackage will be available on PyPI soon.--install the package with pip:--pipinstallgpt4api_dgUsageUsage sample is copied from earlier gpt-3.5-turbo project and is subject to change. It is not yet tested with gpt-4.fromdg_chatgpt_apiimportChatGPTAPIasapiapi.reconnect()api.add_user("user_id","username")api.add_conversation("user_id")message={"role":"user","content":"message"}response=api.get_response("user_id",message)
Package: gpt4-torch
GPT4The open source implementation of the base model behind GPT-4 from OPENAI [Language + Multi-Modal], click here for theResearch PaperInstallationpip install gpt4-torchUsageHere's an illustrative code snippet that showcases GPT-3 in action:importtorchfromgpt4importGPT4# Generate a random input sequencex=torch.randint(0,256,(1,1024)).cuda()# Initialize GPT-3 modelmodel=GPT4()# Pass the input sequence through the modeloutput=model(x)MultiModal IterationPass in text and and image tensors into GPT4importtorchfromgpt4.gpt4importGPT4MultiModal#usageimg=torch.randn(1,3,256,256)text=torch.randint(0,20000,(1,1024))model=GPT4MultiModal()output=model(text,img)📚 Trainingfromgpt4importtraintrain()For further instructions, refer to theTraining SOP.Set the environment variables:ENTITY_NAME: Your wandb project nameOUTPUT_DIR: Directory to save the weights (e.g.,./weights)MASTER_ADDR: For distributed trainingMASTER_PORTFor master port distributed trainingRANK- Number of nodes servicesWORLD_SIZENumber of gpusConfigure the training:Accelerate ConfigEnable Deepspeed 3Accelerate launch train_distributed_accelerate.pyFor more information, refer to theTraining SOP.
Package: gptables
gptablesis an opinionated python package for spreadsheet production.
It produces.xlsxfiles from yourpandasdataframes or usingreticulatein R. You define the mapping from your data to elements of the
table. It does the rest.gptablesuses the officialguidance on good practice spreadsheets.
It advocates a strong adherence to the guidance by restricting the range of operations possible.
The default themegpthemeshould accommodate most use cases.
However, theThemeobject allows development of custom themes, where other formatting is required.gptablesis developed and maintained by theAnalysis Function. It can be
installed fromPyPIorGitHub. The source code is maintained on GitHub.
Users may also be interested ina11ytables, an R native equivalent togptables, andcsvcubed, a package for turning data and metadata into
machine-readable CSV-W files.5 Simple StepsYou map your data to the elements of aGPTable.You can define the format of each element with a customTheme, or simply use the default - gptheme.Optionally design aCoverpage to provide information that relates to all of the tables in your Workbook.Optionally upload anotes_tablewith information about any notes.Youwrite_workbookto win.