Saravanan666/pypi-org huggingface dataset:
Package: language-check
Python wrapper for LanguageTool.This is a fork ofhttps://bitbucket.org/spirit/language_toolthat produces more easily parsable
results from the command-line.Example usageFrom the interpreter:>>> import language_check
>>> tool = language_check.LanguageTool('en-US')
>>> text = u'A sentence with a error in the Hitchhiker’s Guide tot he Galaxy'
>>> matches = tool.check(text)
>>> len(matches)
2Check out someMatchobject attributes:>>> matches[0].fromy, matches[0].fromx
(0, 16)
>>> matches[0].ruleId, matches[0].replacements
('EN_A_VS_AN', ['an'])
>>> matches[1].fromy, matches[1].fromx
(0, 50)
>>> matches[1].ruleId, matches[1].replacements
('TOT_HE', ['to the'])Print aMatchobject:>>> print(matches[1])
Line 1, column 51, Rule ID: TOT_HE[1]
Message: Did you mean 'to the'?
Suggestion: to the
...Automatically apply suggestions to the text:>>> language_check.correct(text, matches)
'A sentence with an error in the Hitchhiker’s Guide to the Galaxy'From the command line:$ echo 'This are bad.' > example.txt

$ language-check example.txt
example.txt:1:1: THIS_NNS[3]: Did you mean 'these'?InstallationTo install via pip:$ pip install --upgrade language-checkIf you are using Python 2, you’ll need to install 3to2 beforehand:$ pip install --upgrade 3to2PrerequisitesPython 3.3+(or 2.7)lib3to2(if installing for Python 2)LanguageTool(Java 6.0+)The installation process should take care of downloading LanguageTool (it may
take a few minutes). Otherwise, you can manually downloadLanguageTool-stable.zipand unzip it
into where thelanguage_checkpackage resides.Vim pluginTo use language-check in Vim, installSyntasticand use the following
settings:letg:syntastic_text_checkers=['language_check']letg:syntastic_text_language_check_args='--language=en-US'Customize your language as appropriate.
Package: languagecodes
languagecodesThis library helps to normalise the ISO 639 codes used to describe languages from
two-letter codes to three letters, and vice versa.importlanguagecodesassert'eng'==languagecodes.iso_639_alpha3('en')assert'eng'==languagecodes.iso_639_alpha3('ENG ')assert'en'==languagecodes.iso_639_alpha2('ENG ')You can install the library from the Python package index:pipinstalllanguagecodesUses data from:https://iso639-3.sil.org/See also:https://www.loc.gov/standards/iso639-2/php/code_list.php
Package: language-data
language_data: a supplement tolangcodesThis package is not meant to be used on its own. Please seelangcodesfor documentation.language_datais a supplement to the langcodes module, for working with standardized codes for human languages. It stores the more bulky and hard-to-index data about languages, particularly what they are named in various languages.For example, this stores the data that tell you that the code "en" means "English" in English, or that "francés" is the Spanish (es) name for French (fr).The functions and test cases for working with this data are inlangcodes, because working with the data correctly requires parsing language codes.DataThe data included in this package is:The names of various languages, in various languagesThe estimated population that speaks each languageThe estimated population that writes each languageThese are all extracted from the UnicodeCLDRdata package, version 40, plus a few additional language names that fill in gaps in CLDR.CaveatsThe estimates for "writing population" are often overestimates, as described in theCLDR documentation on territory data. In most cases, they are derived from published data about literacy rates in the places where those languages are spoken. This doesn't take into account that many literate people around the world speak a language that isn't typically written, and write in adifferentlanguage.The writing systems of Chinese erase most (but not all) of the distinctions between spoken Chinese languages. You'll see separate estimates of the writing population for Cantonese, Mandarin, Wu, and so on, even though you'll likely consider these all to bezhwhen written.CLDR doesn't have language population data for sign languages. Sign languages end up with aspeaking_population()andwriting_population()of 0, and I suppose that is literally true, but there's no data from which we could provide asigning_population()method.Dependencieslanguage_datahas a dependency on themarisa-triepackage so that it can load a compact, efficient data structure for looking up language names.Installationlanguage_datais usually installed as a dependency oflangcodes, and doesn't make much sense without it. You canpip install language_dataanyway if you want.To install thelanguage_datapackage in editable mode, runpoetry installin the package root. (This is the equivalent ofpip install -e ., which will hopefully become compatible again soon via PEP 660.)
Package: languagedet
What is this packageThelanguagedetpackage implements language detection using stopwords and trigrams. It has three clases:languagedet.stopwords.StopWordsDetector: detects language using stopword lists.languagedet.textcat.TextCatDetector: uses the libexttexcat library for language detection.languagedet.mixed.MixedDetector: uses StopWordsDetector and if it fails then use TextCatDetector.IntallationThis package depends on the libexttextcat library. To install it in Ubuntu:$ sudo apt-get install build-essential python-dev libexttextcat-devNow you can install using pip:$ pip install languagedetExampleIn [1]: from languagedet.mixed import MixedDetector

In [2]: det = MixedDetector()

In [3]: det.available
Out[3]:
['fr',
 'en',
 'de',
 'it',
 'da',
 'fi',
 'hu',
 'es',
 'ru',
 'nl',
 'pt',
 'no',
 'tr',
 'sv']

In [4]: det('biblioteca para la detectar idioma')
Out[4]: 'es'ChangelogVersion 0.1.1Modified setup.py.Added a README.txt.Added a MANIFEST.in to include data files missing in version 0.1.Removed dependencies form cython and setuptools-cython.Version 0.1Initial version.Support for language detectión using stopwords and the exttextcat library.
Package: language-detector
language-detectorlanguage-detector detects the language of textInstallationpip install language-detectorPython VersionWorks with both Python 2 and 3Usefrom language_detector import detect_language
text = "I arrived in that city on January 4, 1937"
language = detect_language(text)
# prints EnglishFeaturesLanguages SupportedArabicEnglishFarsiFrenchGermanKurmanci (Kurdish)MandarinRussianSorani (Kurdish)SpanishTurkishTestingTo test the package runpython -m unittest language_detector.tests.testComparisonTest is a comparison of how well language-detector and langid identify languages in thedata sources.packagelanguage-detectorlangidtest-duration (in seconds)0.103.83accuracy96.77%67.74%Excluding LanguagesIf you don't want language-detector to look for certain languages, you can monkey-patch the code.  For example, in order to exclude English:import language_detector
language_detector.char_language = [cl for cl in char_language if cl[1] != "English"]

# proceed as normalDatasetsThe following is a list of datasets used for each language:LanguageDatasetsArabicUN CorporaEnglishUN CorporaFarsiBBC News PersianFrenchUN CorporaGermanDeutsche WelleKurmanci (Kurdish)RudawMandarinUN CorporaRussianUN CorporaSorani (Kurdish)RudawSpanishUN CorporaTurkishBBC News TürkçeContributingIf you'd like to contribute a new language, please consultCONTRIBUTING.mdSupportContact the package author, Daniel J. Dufour, atdaniel.j.dufour@gmail.com
Package: languageflow
LanguageFlowData loaders and abstractions for text and NLPRequirementsInstall dependencies$pipinstallfuture,tox$pipinstallpython-crfsuite==0.9.5$pipinstallCython$pipinstall-Ufasttext--no-cache-dir--no-deps--force-reinstall$pipinstallxgboost==0.82Installation$pipinstalllanguageflowComponentsTransformers: NumberRemover, CountVectorizer, TfidfVectorizerModels: SGDClassifier, XGBoostClassifier, KimCNNClassifier, FastTextClassifier, CRFDataDownload a dataset usingdownloadcommand$languageflowdownloadDATASETList all dataset$languageflowlistDatasetsThe datasets module currently contains:Tagged: VLSP2018-NER, VTB-CHUNK*, VLSP2016-NER*, VLSP2013-POS*, VLSP2013-WTK*Categorized: AIVIVN2019_SA*, VLSP2018_SA*, UTS2017_BANK, VLSP2016_SA*, VNTCPlaintext: VNESES, VNTQ_SMALL, VNTQ_BIGCaution (*): With closed license dataset, you must provide URL to downloadExampleDownloadUTS2017_BANKdataset$languageflowdownloadUTS2017_BANKUseUTS2017_BANKdataset>>>fromlanguageflow.data_fetcherimportDataFetcher,NLPData>>>corpus=DataFetcher.load_corpus(NLPData.UTS2017_BANK_SA)>>>print(corpus)CategorizedCorpus:1780train+197dev+494testsentencesHistory1.1.7 (2018-04-12)Automatic deploy with travis and pypiFix dependencies hell1.1.6 (2017-12-26)Add data module to handle data downloading and data preprocessingAdd many new models: SGDClassifier, XGBoostClassier, FastTextClassifier, CRFAdd new feature: LanguageBoardAutomatic continuous integration with travis-ciBuild docs with readthedocs.org1.1.5 (2017-12-11)Refactor project to integrate with underthesea experiment0.1.0 (2017-09-18)First release on PyPI.
Package: language-local
python-package-templateTo create local package and remote package layers (not to create GraphQL and REST-API layers)directory structureRoot directory should have only .github/, ,gitignore, README.md and the project directory (i.e. location_local_python_package same as repo) - This will allow is to easily switch to mono repo/location_local/location_local/get_country_name/location_local/get_country_name/src/location_local/get_country_name/src/get_country_name.py/location_local/get_country_name/tests/location_local/get_country_name/tests/get_country_name_test.pydatabase Python scripts in /db folderPlease place .py in /dbNo need for seperate file for _ml tablePlease delete the example file if not neededCreate the files to create the database schema, tables, view and populate metadata and Test-Data/db/.py - CREATE SCHEMA ... CREATE TABLE ... CREATE VIEW ... including _ml_table/db/_insert.py to create metadata and Test-Data recordsUpdate the setup.py (i.e.name, version)Please create test directory inside the directory of the project i.e. //testsUpdate the serverless.yml in the root directoryprovider:
stage: play1Update the endpoints in serverless.ymlWorking with VS CodePlease make sure you push to the repo launch.json fie that enables to run and debug the codeUnit-TestWe prefer using pytest and not unittest packagePlease create pytest.init in the project-directory and not in the root directory[pytest]
markers =
    test: custom mark for testsWhen you took care of all the TODOs in the repo, using our infrastructure classes (i.e. Logger, Database, Url, Importer ...) your Feature Branch GitHub Actions Workflow is Green without warnings, all tests are running in GHA, code is documented, README.md is clean and self explained, test coverage is above 90% and all your lines are covered with Unit-Tests, you can filter and analyse your records in Logz.io, pull dev to your Feature Branch and only then create Pull Request to dev
Package: language_middleware
# WSGI Language Middleware## Why?This Wsgi middleware, takes the language from urls similiar to "/es/documents" or from the Accept language header.## Instalation### Prerequisites:* Python >= 2.6* Ludibrio (for the tests)### Installpython setup.py install## Testpython setup.py -q test## Usage### Basicfrom language_middleware import LanguageMiddlewareapplication = my_wsgi_app()application = LanguageMiddleware(application, default_language = 'es', valid_languages = ('en', 'es', 'fr'))Call to the url **/fr/documents**, and my_wsgi_app will receive:environ['HTTP_ACTIVE_LANGUAGE'] = 'fr'Call to the url **/documents/1**, and my_wsgi_app will receive:environ['HTTP_ACTIVE_LANGUAGE'] = 'es'### Clean URLThe clean_url option, removes the language info from the URL.from language_middleware import LanguageMiddlewareapplication = my_wsgi_app()application = LanguageMiddleware(application, default_language = 'es', valid_languages = ('en', 'es', 'fr'), clean_url = True)Call to the url **/fr/documents**, and my_wsgi_app will receive:environ['PATH_INFO'] = '/documents'environ['HTTP_ACTIVE_LANGUAGE'] = 'fr'Call to the url **/documents/1**, and my_wsgi_app will receive:environ['PATH_INFO'] = '/documents/1'environ['HTTP_ACTIVE_LANGUAGE'] = 'es'### With localeIf you have the following locale directory/home/user/locale/es/LC_MESSAGES/hello.po/home/user/locale/es/LC_MESSAGES/hello.mo/home/user/locale/en/LC_MESSAGES/hello.po/home/user/locale/en/LC_MESSAGES/hello.moYou can set an option to active the language, check the test from more info:+IMPORTANT: **Default language locale path must exists, if not last successfully used locale will be used (if any)**from language_middleware import LanguageMiddlewareapplication = my_wsgi_app()application = LanguageMiddleware(application,default_language = 'es',valid_languages = ('en', 'es', 'fr')clean_url = True,locale_path = '/home/user/locale',locale_name = 'hello')### Set Script NameStarting with wsgi-language-middleware 0.4 you can pass **set_script_name** to **LanguageMiddleware**, it will set the **SCRIPT_NAME** to a slash followed by the chosen language (e.g. **/es**)This can help you if your application uses url-creation helpers (e.g. pylons **h.url()**), which do rely on the script name and should automatically prepend links with the given locale.### Switching off global gettext translation installationIf you want to take care of the locale loading yourself (e.g. by inspecting environ['HTTP_ACTIVE_LANGUAGE'] later on) or do not use gettext, you can suppress the installation of the gettext translation by using **do_gettext_install=False** in **LanguageMiddleware()** .
Package: languager
LanguagerISO639 Language ServiceThis module identifies languages from ISO639-1 and ISO639-3 codes or ISO names and provides a convenient class to access
related attributes. It is also possible to lookup codes for languages based on their ISO names. However, the name based
lookup will be slower as all language names are compared in lowercase for that.Basic flows:Code is 2 charactersLookup long codeLookup dataCode is 3 charactersLookup dataCode is something elseLoop all language names to check for matchinput_language == language_name.lower()Lookup dataThis means that that Name based lookup isn-times slower than the other two options. But this really should not make a
difference.DataThe data is taken fromiso639-3.sil.organd is stored in thetablesfolder. Further releases will update these tables.CodeThe code ingenerator.pygenerates a single python file that contains all lookup tables and
methods.Languagecode: The ISO639-3 Codeshort: ISO639-1 Code if availabledeprecated: True if the definition is deprecatedmacro: True if this is in a macrolanguage gropupparent: The parent macrolanguagemacros: Any languages belonging to this macrolanguageExamplesChecking the macrolanguages for Chinese:fromlanguagerimportget_languagelang=get_language('zho')# lang = get_language('zh')# lang = get_language('Chinese')# lang = get_language('does not exist', default='zho')forlanguageinlang.macros:print(language)# czo# csp# yue# cnp# cmn# czh# hak# nan# wuu# cjy# lzh# gan# mnp# cpx# hsn# cdo
Package: languageslayer
# Language-Slayer
A flexible and easy-to-use python library for detecting languages.
Can detect over 250 languages.A python library built for the [LanguageLayer API](https://languagelayer.com).<br>Visit the website and register for an acceskey for free. After that, you can make 3000+ requests.<br>###Features[x] Super Fast.(Under good internet conditions, it returns an answer under 3 seconds)<br>[x] Very Accurate. (I personally rate it 9/10)<br>[x] Easy-to-Use <br>[x] Flexible <br>[x] Light-weight <br>[x] Reduces the amount of bytes needed to retreive info from server <br>[x] Developer Friendly <br><hr>Simple, flexible and easy to use.(Contains only 4 methods)<br>`access_key()`- This method is used to lock-on your accesskey.<br>`analysis()`- This method returns a full analysis of the sentence you are detecting.<br>`detect()`- Self-explanatory; this method is used to detect the language.<br>`json()`- This function returns a json for devs to analyse.<br>
Package: languagestring
languagestringTo use:from languagestring import languagestringls = languagestring()# get czech charactersls.language_string('cz',case='lowercase')'áčďéěíňóřšťúůýž'Working Languages:{
	"se" : "swedish",
	"fr" : "french",
	"de" : "german",
	"nl" : "dutch",
	"es" : "spanish",
	"pt" : "portuguese",
	"pl" : "polish",
	"cz" : "czech",
	"it" : "italian"
}Methods:`language_string`- returns a string of the language characters.`language_array`- returns a list of the language characters.`alt_codes`- returns a list of the language alt codes.`description`- returns a list of the language description.Arguments:`case` is defaulted to all which returns lowercase, uppercase, and other characters.`language` is the two letter iso that is the working language.
Package: language-tags
This Python API offers a way to validate and lookup languages tags.StandardIt is based onBCP 47(RFC 5646) and the latestIANA language subtag registry.This project will be updated as the standards change.Language-tags 1.0.0 only supports Python 3, for Python 2 support, please use language-tags 0.5.0.JSON dataSee thelanguage-subtag-registryproject for the underlying JSON data.Javascript versionThis project is a Python version of thelanguage-tagsJavascript project.
Package: language_tool
Example usage>>> import language_tool
>>> lang_tool = language_tool.LanguageTool("en-US")
>>> text = "A sentence with a error in the Hitchhiker’s Guide tot he Galaxy"
>>> matches = lang_tool.check(text)
>>> len(matches)
2Check out someMatchobject attributes:>>> matches[0].fromy, matches[0].fromx
(0, 16)
>>> matches[0].ruleId, matches[0].replacements
('EN_A_VS_AN', ['an'])
>>> matches[1].fromy, matches[1].fromx
(0, 50)
>>> matches[1].ruleId, matches[1].replacements
('TOT_HE', ['to the'])Print aMatchobject:>>> print(matches[1])
Line 1, column 51, Rule ID: TOT_HE[1]
Message: Did you mean 'to the'?
Suggestion: to the
... with a error in the Hitchhiker’s Guide tot he Galaxy
                                           ^^^^^^Automatically apply suggestions to the text:>>> language_tool.correct(text, matches)
'A sentence with an error in the Hitchhiker’s Guide to the Galaxy'InstallationTo install the package for Python 3, use:$ ./setup.py installTo install the package for Python 2, use:$ python2 setup.py installOn Windows, you may use one of the MSI binary packages provided on thedownload page.PrerequisitesPython 3.2+(or 2.7)LanguageToollib3to2(if installing for Python 2)The installation process should take care of downloading LanguageTool
(it may take a few minutes).
Otherwise, you can manually downloadLanguageTool-stable.zipand unzip it into where thelanguage_toolpackage resides.LanguageTool requires Java 6 or later.
Package: langumo-ko
langumo-ko한국어 말뭉치용 langumo parser 모음Introductionlangumo-ko는langumo라이브러리에서 사용할 수 있는 한국어용Parser를 제공합니다.langumo를 사용하여
데이터셋을 빌드할 시, 해당 라이브러리에서 구현된Parser를 사용하여 다양한 말뭉치
데이터를 간단하게 사용할 수 있습니다.langumo-ko가 지원하는 말뭉치 종류는 다음과
같습니다.나무위키(langumo_ko.NamuWikiParser):
나무위키 덤프 파일을 파싱합니다. 7z 형식으로 압축된 덤프 파일에 포함된 원본 json
파일을 사용해야 합니다.모두의 말뭉치- 신문 말뭉치
(langumo_ko.ModuNewsParser): 모두의 말뭉치에서 제공하는 신문 말뭉치
데이터를 파싱합니다.모두의 말뭉치- 문어 말뭉치
(langumo_ko.ModuWrittenParser): 모두의 말뭉치에서 제공하는 문어
말뭉치 데이터를 파싱합니다.모두의 말뭉치- 웹 말뭉치
(langumo_ko.ModuWebParser): 모두의 말뭉치에서 제공하는 웹
말뭉치 데이터를 파싱합니다.InstallationWith piplangumo-ko는PyPI 저장소에 배포되어 있습니다.pip을
이용하여 다음과 같이 설치할 수 있습니다.$pipinstalllangumo-koFrom sourcepip을 사용하는 대신, 해당 레포지토리를 내려받고 직접 빌드해 설치할 수 있습니다.$gitclonehttps://github.com/affjljoo3581/langumo-ko.git
$cdlangumo-ko
$pythonsetup.pyinstallUsagelangumo로 위에서 나열한 말뭉치를
빌드하기 위해서 다음과 같이build.yml을 수정하면 됩니다.langumo:inputs:-path:src/NIKL_NEWSPAPER(v1.0).zipparser:langumo_ko.ModuNewsParser# other configurations...Licenselangumo-ko라이브러리는Apache-2.0 라이센스를 갖습니다.
Package: langvec
Language of Vectors (LangVec) is a simple Python library designed for transforming numerical vector data into a language-like structure using a predefined set of words (lexicon).ApproachLangVecpackage leverages the concept of percentile-based mapping to assign words from a lexicon to numerical values, facilitating intuitive and human-readable representations of numerical data.Where to use LangVecTheLangVeclibrary finds application in semantic search and similarity based systems, where understanding the proximity between vectors is crucial.By transforming complex numerical vectors into a lexicon-based representation,LangVecfacilitates an intuitive understanding of these similarities for humans.This transformation is particularly advantageous in scenarios where interpreting raw numerical data (like floating points or integers) can be challenging or less informative.
In fields like machine learning and natural language processing,LangVeccan assist in tasks such as clustering or categorizing data, where a human-readable format is preferable for quick insights and decision-making.InstallationpipinstalllangvecUsageimportnumpyasnpimportlangvec# Initialize LangVeclv=LangVec()# Generate some random datadata=np.random.rand(100)# Fit to this data (getting know to distribution)lv.fit([data])# Example vector for predictioninput_vector=np.random.rand(15)# Make prediction on unseen vector embeddingprint(lv.predict(input_vector))
Package: langwatch
LangWatch Python SDKGo tohttps://langwatch.aito setup your account.To trace OpenAI calls:from openai import OpenAI+ import langwatch.openaiclient = OpenAI()+ with langwatch.openai.OpenAITracer(client):completion = client.chat.completions.create(model="gpt-3.5-turbo",messages=[{"role": "system","content": "You are a helpful assistant that only reply in short tweet-like responses, using lots of emojis.",},{"role": "user", "content": message},],stream=True,)To trace LangChain agent:+ import langwatch.langchain# ...chain = LLMChain(llm=ChatOpenAI(),prompt=chat_prompt,output_parser=CommaSeparatedListOutputParser(),)+ with langwatch.langchain.LangChainTracer() as langWatchCallback:-   result = chain.run(text="colors")+   result = chain.run(text="colors", callbacks=[langWatchCallback])
Package: langwhat
LangWhatAnswer "What is it?" on the command line with the power of large language models (LLMs).pyWhatLLM version, leveraging OpenAI API and Sydney (Bing AI Chat).LangWhatScreenshotsOpenAISydneyCachingFeaturesInstallationpipxpipUsageDevelopCreditsScreenshotsOpenAIlangwhat'f7316ffccd4d2d555a7522328cf792dd73bfbcd9'langwhat'f7316ffccd4d2d555a7522328cf792dd73bfbcd9'--zhSydneySydney fixed my type "marry" automatically.langwhat'marry ball washington'-s

langwhat'marry ball washington'-s-zCachingResponses are much faster when cache hits, and token usage becomes 0.Note that Sydney doesn't support counting token usage, and always shows 0.lwteddy--show-token-usage

lwteddy--show-token-usageFeaturesUses few-shot prompting to reduce model mis-behaviorEnglish by default for superior response speed and accuracySupports supplying API Key via either environment variable or config file at~/.config/langwhat/api_key.txtSupports usingSydneywith-sflagCaching responses in a local sqlite databaseInstallationpipxThis is the recommended installation method.$ pipx install langwhat

# python 3.11 or higher is required, if your pipx uses a lower version of python by default,
# you could run the following command to install langwhat with python 3.11
# pipx install --python "$(which python3.11)"pip$ pip install langwhatUsage$ langwhat --help

usage: lw [-h] [-z] [-s] [-C] [--show-token-usage] [-V] what

positional arguments:
  what                what is it

options:
  -h, --help          show this help message and exit
  -z, --zh            Use Mandarin to prompt and answer
  -s, --sydney        Use Sydney (Bing AI) instead of OpenAI
  -C, --no-cache      Disable cache
  --show-token-usage  Show token usage
  -V, --version       show program's version number and exitDevelop$ git clone https://github.com/tddschn/langwhat.git
$ cd langwhat
$ poetry installCreditsLangChainEdgeGPTYihong