Saravanan666/pypi-org huggingface dataset:
Package: langlearncopilot
LangLearnCopilotLangLearnCopilot is a collection of functions and tools, to generate content in a proper format, to help you learn a new language.For now, I am focusing on French (my personal interesting) and Spanish (for friends).While this is a standalone library, my advice is, for the best results, to use its outcomes hand-in-hand with the awesome flashcards application,AnkiThe main two applications at the momentInstallationpip install langlearncopilotUsageYou can find examples in the./examplesfolderNote: you need to set your OpenAI key first. If theOPENAI_API_KEYis declared already as an environment variable, or exisiting in the current folder in a.envfile, then the package will find it.Other wise, you can set it manually by callingfromlanglearncopilot.llm_callsimportset_openai_keyset_openai_key("ENTER KEY VALUE HERE")Given a text, extract all the unique words and its translation from that textfromlanglearncopilotimportgeneratorsgenerators.generate_unique_words(article="Bonjour, je m'appelle Jean. Je suis un étudiant à l'université de Paris.")returns{
    'bonjour': 'hello',
    'je': 'i',
    "m'appelle": 'is called',
    'suis': 'am',
    'un': 'a',
    'étudiant': 'student',
    'à': 'at',
    "l'université": 'university',
    'de': 'of'
}Given a word, generate 3 phrases that are using these wordsfromlanglearncopilotimportgeneratorsgenerators.generate_phrases("combien")returns[{'combien':{'phrase':'Combien de personnes sont venues à la fête?','translation':' How many people came to the party?'}},{'combien':{'phrase':'Combien coûte ce sac à dos?','translation':' How much does this backpack cost?'}},{'combien':{'phrase':"Il m'a demandé combien de temps cela prendrait.",'translation':' He asked me how long it would take.'}}]Extract unique words from URL, with their translationfromlanglearncopilot.parsersimportget_text_from_webpagefromlanglearncopilot.generatorsimportgenerate_unique_wordsdefmain():# Get text from a webpagetext=get_text_from_webpage(url="https://www.lemonde.fr/planete/article/2023/08/27/comment-les-parcs-nationaux-americains-tentent-de-faire-face-aux-effets-du-rechauffement-climatique_6186696_3244.html")# Generate unique words from the textwords=generate_unique_words(article=text,language="french")print(words)if__name__=="__main__":main()returns{'comment':'how','les':'the','parcs':'parks','nationaux':'national','américains':'american','tentent':'try','de':'of',...}
Package: lang-localization
#lang-localization
With lang-localization you can translate your program to different languages.
It is not possible translating your program easier.How to useStart by creating a folder named "localization" in the directory where your python file is located. For every language you want to add you need to create a file with the .loc ending in the new directory. For example "de.loc" or "en.loc".
The inside of the file must look like this:hw = Hello World!
button1 = Press MeYour python file must look like this:importlocalizationlocalization.setLang("de")# The string must be the same as the filename of your translation without .locprint(localization.get("hw"))randomButtonCreateFunction(text=localization.get("button1"))
Package: langma
Awesome NLP framework you are waiting for ✨langmawas the short name ofQomolangma, it's the highest mountain on Earth
Snow-capped mountains, azure sky, golden rays of sunshine, the pure and
mysterious aura of Tibet has excited the imagination of so many people of the world.Installation⚙ 🔑 🎛️ 🙈 🎠 ❌ 🎉pipinstalllangmaRight now...langmais now on your environment, enjoy...if you don't know what to do and how, enterlangma -hfor help on
terminalREFERENCESloglyKashgari
Package: langmanager
LangManagerthis library allows you to create translations of projects into other languagesinstall:installing with pypipip install langmanagerinstalling with gitgit clone https://github.com/ASVIEST/LangManager.gitSimple example:fromlangmanagerimport*language=input('language:   ')lang(language)trans=translate_get('hello world')print(lan())print(trans)language file(en):'hello world':'hello world';And language file(ru):'hello world':'привет мир';Language standard file name - lan(ISO 639-1).txt examples: en.txt, ru.txt, zh.txt
But file name can change through function filepath_en , filepath_ru and others
Language can be changed during workingImproved example:fromlangmanagerimport*language=input('language:   ')lang(language)trans=translate_get('hello world')print(lan())print(trans)lang('ru')filepath_ru('en.txt')trans=translate_get('hello i')print(trans)'hello world':'hello world';
'hello i':'hello i';And language file(ru):'hello world':'привет мир';
'hello i':'привет я';
Package: langmo
The library for distributed pretraining and finetuning of language models.Supported features:vanilla pre-training of BERT-like modelsdistributed training on multi-node/multi-GPU systemsbenchmarking/finetuning the following tasksall GLUEMNLI  + additional validation on HANSmore coming soonusing siamese architectures for finutuningPretrainingPretraining a model:mpirun -np N python -m langmo.pretraining config.yamllangmo saves 2 types of snapshots: in pytorch_ligning formatTo resume crashed/aborted pretraining session:mpirun -np N python -m langmo.pretraining.resume path_to_runFinetuning/EvaluationFinetuning on one of the GLUE tasks:mpirun -np N python -m langmo.benchmarks.GLUE config.yaml glue_tasksupported tasks:cola, rte, stsb, mnli, mnli-mm, mrpc, sst2, qqp, qnliNLI task has additional special implentation which supports validation on adversarial HANS dataset,
as well as additional staticics for each label/heuristic.To perfrorm fibetuning on NLI run as:mpirun -np N python -m langmo.benchmarks.NLI config.yamlFinetuning on extractive question-answering tasks:mpirun -np N python -m langmo.benchmarks.QA config.yaml qa_tasksupported tasks:squad, squad_v2example config file:model_name: "roberta-base"
batch_size: 32
cnt_epochs: 4
path_results: ./logs
max_lr: 0.0005
siamese: true
freeze_encoder: false
encoder_wrapper: pooler
shuffle: trueAutomatic evaluationlangmo supports automatic scheduling of evaluation runs for a model saved in a given location, or for all snapshots found int /snapshots folder.
To configure langmo the user has to create the following file:./configs/langmo.yaml with entry “submit_command” correspoding to a job submission command of a given cluster. If the file is not present, the jobs will not be submitted to the job queue, but executed immediately one by one on the same node../configs/auto_finetune.inc - the content of this file will be copied to the beginning of the job scripts. Place here directive for e.g. slurm job scheduler such as
which resource group to use, how many nodes to allocate, time limit etc. Set up all necessary environment variables, particulalry NUM_GPUS_PER_NODE and
PL_TORCH_DISTRIBUTED_BACKED (MPI, NCCL or GLOO). Finally add mpirun command with necessay option and end the file with new line.
Command to invoke langmo in the right way will be added automatically../configs/auto_finetune.yaml - any parameters such as batch size etc to owerride the defaults in a fine-tuning run.To schedule evaluation jobs run from the login node:python -m langmo.benchmarks path_to_model task_namethe results will be saved in the eval/task_name/run_name/ subfolder in the same folder the model is saved.Fugaku notesAdd these lines before thereturnof_compare_versionstatement ofpytorch_lightning/utilities/imports.py.:if str(pkg_version).startswith(version):
    return TrueThissedcommand should do the trick:sed -i -e '/pkg_version = Version(pkg_version.base_version/a\    if str(pkg_version).startswith(version):\n\        return True' \
  ~/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py
Package: langmuir
Programmatically accessible current-voltage characteristics for ideal and non-ideal Langmuir probes. See documentation onReadTheDocs.InstallationInstall from PyPI usingpip(preferred method):pip install langmuirOr download the GitHub repositoryhttps://github.com/langmuirproject/langmuir.gitand run:python setup.py installGetting StartedThe Langmuir library contains a collection of functions that compute the current collected by a conductor immersed in a plasma according to various models (characteristics). As an example, consider a 25-mm long cylindrical probe. Below is a comparison of the predicted current according to OML theory and thefinite-length(FL) model which accounts for end effects. From characteristics it is in turn possible to infer plasma parameters using Langmuir probes. See full documentation onReadTheDocs.
Package: langscikw
Keyword extraction from langsci publicationslangscikwis a Python package and command line tool for bigram keyterm extraction. It is optimized for long, English, linguistic publications and can also be applied to TeX code.Keyword extraction is done in three steps. No preprocessing is needed.Step 1:KWE from the input document usingYAKE. This is the simplest step as it doesn't need a corpus and should extract the most important keywords.Step 2:KWE usingTF-IDFtrained on a raw TeX corpus. This step yields some more general keywords relevant to the linguistic discipline.Step 3:KWE using TF-IDF trained on a detexed corpus. This step fills in some missing keywords that also appear in a reference corpus of 10,000 previously accepted keywords,keywordslist.txt.The number of steps can be controlled by (not) providing the relevant corpora during training. The result needs some manual correction and supplementation of relevant unigrams or trigrams.Installationpip3 install langscikwDeveloped in Python 3.7.3 32-bit. Needs at least Python 3.7 and the following packages: jellyfish, joblib, networkx, scikit-learn, segtok, regex.Download the langsci corpus files fromhereor use your own.UsageCommand lineThe command line tool provides only a simple interface. If you'd like to customize the model parameters or the number of steps, please see below.langscikw inputfile [n] [corpus1] [corpus2] [keywordslist] [--silent]The keywords are printed to the console and can be redirected to a text file.Argumentsinputfile: Path to a .txt or .tex file or directory from which to extract keywords.n:OptionalNumber of keywords to extract. Defaults to 300.corpus1:OptionalPath to corpus for step 2, usually raw TeX files or ajoblib-compressedfile. If not provided, looks for corpus_tex.gz in the current directory.corpus2:OptionalPath to corpus for step 3, usually detexed files or ajoblib-compressedfile. If not provided, looks for corpus_detexed.gz in the current directory.keywordslist:OptionalPath to a list of gold keywords for step 3. A default list based on langsci publications is installed with the package.--silent:OptionalOnly print the result to the console, no progress updates.KWE classimportlangscikwinput_path="my_book"# File/directory to extract keywords fromkeywords_path="keywords.txt"# File to save keywords tokwe=langscikw.KWE()kwe.train("corpus_tex.gz","corpus_detexed.gz")kws=kwe.extract_keywords(input_path,n=300,dedup_lim=0.85)forkwinkws:print(kw)# Keywords are alphabetically sorted stringsKeyword argumentsn:OptionalNumber of keywords to extract. Defaults to 300.dedup_lim:OptionalDeduplication limit. Keywords that have a Jaro-Winkler Similarity of >dedup_lim are not added to the final list. Defaults to 0.85.Stand-alone modelsThe YAKE and TF-IDF models may also be used on their own. Please consult the docstrings for more information.importlangscikwyake=langscikw.yakemodel.YakeExtractor()# -> extract_keywords()tfidf=langscikw.tfidfmodel.TfidfExtractor()# -> train() -> extract_keywords()
Package: langsmith-wrappers
LangSmith API WrappersSome functionality to wrap common apis (e.g.,openai) withLangSmithinstrumentation.Example:fromlangsmith.wrappers.openaiimportopenairesult=openai.ChatCompletion.create(model="gpt-3.5-turbo",messages=[{"role":"user","content":"What's the weather like in san francisco right now?",}],functions=[{"name":"get_current_weather","description":"Get the current weather in a given location","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and state, e.g. San Francisco, CA",},"unit":{"type":"string","enum":["celsius","fahrenheit"]},},"required":["location"],},}],)print(result)You can try wrapping your own library. Function calls will be mapped as "chain" runs.Example:fromlangsmith.wrappers.baseimportModuleWrapperimporttransformersastransformers_basetransformers=ModuleWrapper(transformers_base)pipe=transformers.pipeline("text2text-generation",model="google/t5-efficient-tiny")result=pipe("This is a test")
Package: langsnapy
Langsnapy - Tool to snapshot your LLMs, NLP Models and ChatbotsI am looking for better nameIdea behind this tool, is to maintain snapshots of what your model can answer.How it works:You define a set of questions. This tool run this questions against your model and save the answers. Then you can run this questions again and compare the answers with the saved ones.How to usepipinstalllangsnapyCheckexamplesfolder.How to testpoetryinstall-Eruntime-deps--withdev
poetryrunpytest.
poetryrunruff.
Package: langsrc
langSrcInstallpip3installlangSrcDemofromlangSrcimportLanguageDetectorlang=LanguageDetector("zh","lang.json")lang.register("language",{"zh":"语言","en":"Language","jp":"言語","kor":"언어","fra":"Langue","spa":"Idioma","th":"ภาษา",},)print(lang.language)# or print(lang["language"])# 语言This will generate a file namedlang.json:{"language":{"zh":"语言","en":"Language","jp":"言語","kor":"언어","fra":"Langue","spa":"Idioma","th":"ภาษา"}}
Package: lang-support
My python program that allows you to use multiple languages in your programsClass namedLangSupportis in lang_support.py file.Install via pip:pip install lang_supportmain.pyfile, in tests directory, is an example of use.To run example:cdlang_support
py-mtests.mainImportLangSupportclass:import lang_support.lang_supportorfrom lang_support.lang_support import LangSupportImportLangSupportDLclass:import lang_support.lang_support_data_loggerorfrom lang_support.lang_support_data_logger import LangSupportDLgenerate_translation.pyfile is a script which read specified language file and generate translation for selected language (deepl package and token needed)Place DeepL token indata_token.jsonfile (in directory withgenerate_translation.pyfile) like below:{"deepl-token":"your-token"}To run translation generator, enter following commands:cdlang_support
pipinstalldeepl# if not installedpy-mtests.generate_translation.generate_translation.generate_translation--help
Package: langtable
Purpose of this packagelangtable is used to guess reasonable defaults for locale, keyboard,
territory, …, if part of that information is already known. For
example, guess the territory and the keyboard layout if the language
is known or guess the language and keyboard layout if the territory is
already known.LicenseGPLv3+, see the included file “COPYING”.Translations for languages and territory names are from CLDR which is
governed by the Unicode Terms of Use, see the included file
“unicode-license.txt”. The short name for this Unicode license is
“MIT”. See:https://fedoraproject.org/wiki/Licensing:MIT?rd=Licensing/MIT#Modern_Style_without_sublicense_.28Unicode.29InstallationTo install langtable, runmake install DESTDIR=/usrTo create a distribution tarball runmake distTo run the test cases in the source directory:make testHow to use itimport langtableFunctions in the public API:parse_locale()
list_locales()
list_keyboards()
list_common_languages()
list_common_locales()
list_common_keyboards()
list_consolefonts()
list_inputmethods()
list_timezones()
list_scripts()
language_name()
territory_name()
timezone_name()
languageId()
territoryId()
supports_ascii()
list_all_languages()
list_all_locales()
list_all_keyboards()
list_all_territories()
list_all_timezones()
list_all_scripts()
list_all_input_methods()
list_all_console_fonts()Some examples to show the usage are found in the documentation
of the public functions in langtable.py.(Some more examples are in the test cases in the file test_cases.py):
Package: langtojson
LangToJsonTool to convert .lang files to JSONFree software: MIT licenseCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2020-06-06)First release on PyPI.0.1.1 (2020-06-06)Removed python 3.5 from supported versionsRemoved useless test filesAdded encoding0.1.2 (2020-06-06)Fix output files utf-80.1.3 (2020-06-06)Add ensure_ascii=false
Package: lang-trans
# lang-trans[![License](https://img.shields.io/badge/License-Apache_2-0BDA51.svg?style=plastic)](http://www.apache.org/licenses/LICENSE-2.0)[![PyPI](https://img.shields.io/pypi/v/lang-trans.svg?style=plastic)](https://pypi.python.org/pypi/lang-trans)[![Downloads](https://img.shields.io/pypi/dm/lang-trans.svg?style=plastic)](https://pypi.org/project/lang-trans/)[![Python version](https://img.shields.io/pypi/pyversions/lang-trans.svg?style=plastic)](https://pypi.org/project/lang-trans/)[![Travis](https://img.shields.io/travis/kariminf/lang-trans.svg?style=plastic)](https://travis-ci.org/kariminf/lang-trans)[![Codecov](https://img.shields.io/codecov/c/github/kariminf/lang-trans.svg?style=plastic)](https://codecov.io/gh/kariminf/lang-trans)Python transliteration library (mostly from non-latin scripts, such as Arabic, Japanese, etc.)## UseHere an example of using Buckwalter transliteration for Arabic:```python#!/usr/bin/env python# -*- coding: utf-8 -*-from lang_trans.arabic import buckwalterprint buckwalter.transliterate(u'هذا البرنامج يعطينا نطق الحروف')print buckwalter.untransliterate('h*A AlbrnAmj yETynA nTq AlHrwf')```abbreviated methods:- transliterate(text): trans(text)- untransliterate(text): untrans(text)## Available transliterators- arabic- buckwalter- arabtex- iso233- japanese- hepburn- nihonshiki- kunreishiki## Similar projects## LicenseCopyright (C) 2017-2018 Abdelkrime Aries (kariminfo0@gmail.com)Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License at[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.
Package: langtranslator
langtranslatorThis module allows you to translate your program to different languages.
This is done by making a json with all the translations. For further information read the usage chapter.Installingpip install langtranslatorUsageJSON:{
    "info": {"languages": ["en", "ger", "fr"]},
    "translations": {
	    "string1": {"en": "This is a string.", "ger": "Das ist ein String.", "fr": "I dont know frech."},
	    "string2": {"en": "test", "ger": "abc", "fr": "bonjour"}
    }
}In info you specify the languages that are translated.
In translations you specify the different translated strings.Python:How to initialize the module:from langtranslator import langtranslator

translator = langtranslator.langtranslator("filename.json", "language")How to get strings in the specified language:translator.get("stringKey")
Package: langtree
Welcome to Langtree!What is LangtreeLangtree is a micro framework for building natively parallel LLM chains. Our goal is to keep the core of this repo under 100 LOC (not including test cases).Why use Langtree?Langtree is dead simple. It's under 100 LOC for the core of the project. Seriously.Theres only 3 constructs: Operators, Buffers, and Prompts.This means the following capabilities aretrivial:Debuggability: Langtree isinsanelydebuggable.
It's so minimal that the underlying api's are almost identical to those exposed by our integrations.
For example, themessagesfield on theOpenAIChatobject is the exact field that is passed to the underlying api.Natively Parallel: A common issue with LLM tools is parallelism. With Langtree, doing this is a piece of cake. This means Agents, retrieval, etc. are far quicker to iterate upon.Make sure to read the docs before using the parallel toolsLangchain Interoperable: Langtree can be usedwithLangchain. Although we are opinionated about Langtree,Langchain offers a LOT of value.Back Compatibility: Expect Langtree to be extremely stable. Its so flexible that itactually adjustswhen integrations change. It's only this capable because it is minimal. This is why we are commited to keeping the core library under 200 LOC. Forever.FeaturesChain recordingWe offer a simple abstraction to record analytics onanychain(Including Langchain code)We currently only have recording to JSON but expect to see more by 08/11/23ProvidersModels: We currently support OpenAI.RoadmapFunction Calling: We plan to support function calls in the VERY near future.Models: To support Azure, Huggingface (local and remote), Anthropic and CohereVectorDBs: Currently in development. Again we promise to maintain minimal levels of abstraction.Recording: We are in the process of implementing S3, Planetscale, and more!
Package: language-agent
Language-AgentLanguage-Agent is a Python package that allows for easy integration with OpenAI's GPT-4 model. The package provides a simple API to generate text completions given a user prompt.Lanuage-Agent is perfect for developers looking to utilize OpenAI's powerful language model in their applications without the need to directly interact with OpenAI's API.Key FeaturesSimple API for text generationAutomatic handling of API keys through environment variablesEasily customizable user prompt and contextInstallationTo install Lanuage-Agent, you can use pip:pipinstalllanguage_agentBefore using the package, you'll need to set your OpenAI API key in a.envfile:OPENAI_API_KEY='your-key-here'Make sure the.envfile is in the root directory of your project.UsageUsing AI-Agent is simple. Below is a sample code snippet:fromdotenvimportload_dotenvfromlanguage_agent.promptimportagent_promptload_dotenv()defmain():context='write the cotext i.e. you are a reviewer of an exam'user_input='Write your question to match the context'response=agent_prompt(context,user_input)print("Agent's response:",response)if__name__=="__main__":main()In this example, the context and user input are combined and sent to OpenAI's GPT-4 model, which generates a text completion. The text completion is then printed to the console.LicenseLanguage-Agent is licensed under the terms of the MIT License.Please note that using Language-Agent requires an API key from OpenAI, which may be subject to additional terms of use. Users are responsible for securely managing their API keys by storing them in a .env file and not sharing or exposing them in public or unsecured areasrepository:https://github.com/Andi5986/language_agent
Package: languageassistant
LanguageAssistantDocumentation:https://dagleaves.github.io/languageassistantSource Code:https://github.com/dagleaves/languageassistantPyPI:https://pypi.org/project/languageassistant/LanguageAssistant is a large language model (LLM) powered foreign language learning assistant. This harnesses the
language capabilities of LLMs to provide verbal communication capabilities focused on learning foreign languages.This project is largely a proof of concept, but has been setup to be extensible so that it could be used as a
backend for a full-fledged application utilizing a better UI, microphone recording, transcription, LLM prompts,
and/or multi-lingual text-to-speech.Features of LanguageAssistant:Full real-time vocal multilingual conversationsReal-time voice transcriptionLLM text responsesMultilingual text-to-speechLLM agent for lesson planningInstallationpipinstalllanguageassistant