Saravanan666/pypi-org huggingface dataset:
Package: toil-http-parser
http-parserHTTP request/response parser for Python compatible with Python 2.x
(>=2.6), Python 3 and Pypy. If possible a C parser based onhttp-parserfrom Ryan Dahl will be used.http-parser is under the MIT license.Project url:https://github.com/benoitc/http-parser/Requirements:Python 2.6 or sup. Pypy latest version.Cython if you need to rebuild the C code (Not needed for Pypy)Installation$ pip install http-parserOr install from source:$ git clone git://github.com/benoitc/http-parser.git
$ cd http-parser && python setup.py installNote: if you get an error on MacOSX try to install with the following
arguments:$ env ARCHFLAGS=”-arch i386 -arch x86_64” python setup.py installUsagehttp-parser provide youparser.HttpParserlow-level parser in C that
you can access in your python program andhttp.HttpStreamproviding
higher-level access to a readable,sequential io.RawIOBase object.To help you in your day work, http-parser provides you 3 kind of readers
in the reader module: IterReader to read iterables, StringReader to
reads strings and StringIO objects, SocketReader to read sockets or
objects with the same api (recv_into needed). You can of course use any
io.RawIOBase object.Example of HttpStreamex:#!/usr/bin/env python
import socket

from toil_http_parser.http import HttpStream
from toil_http_parser.reader import SocketReader

def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    try:
        s.connect(('gunicorn.org', 80))
        s.send("GET / HTTP/1.1\r\nHost: gunicorn.org\r\n\r\n")
        r = SocketReader(s)
        p = HttpStream(r)
        print p.headers()
        print p.body_file().read()
    finally:
        s.close()

if __name__ == "__main__":
    main()Example of HttpParser:#!/usr/bin/env python
import socket

# try to import C parser then fallback in pure python parser.
try:
    from toil_http_parser.parser import HttpParser
except ImportError:
    from toil_http_parser.pyparser import HttpParser


def main():

    p = HttpParser()
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    body = []
    try:
        s.connect(('gunicorn.org', 80))
        s.send("GET / HTTP/1.1\r\nHost: gunicorn.org\r\n\r\n")

        while True:
            data = s.recv(1024)
            if not data:
                break

            recved = len(data)
            nparsed = p.execute(data, recved)
            assert nparsed == recved

            if p.is_headers_complete():
                print p.get_headers()

            if p.is_partial_body():
                body.append(p.recv_body())

            if p.is_message_complete():
                break

        print "".join(body)

    finally:
        s.close()

if __name__ == "__main__":
    main()You can find more docs in the code (or use a doc generator).Copyright2011-2013 (c) Benoît Chesneau <benoitc@e-engura.org>
Package: toisan-lang
toisan-langA programming language based on Toisanese, aka Taishanese, a dialect of CantoneseInstallpip install toisan-langCommand Line Usagepython -m toisan_lang "..."to transpile and execute toisan-lang in Python--show xxxto show extra information if given:treefor parse tree,codefor transpiled python codeFor example,Transpile and executepython -m toisan_lang "你系嚿叉烧。你个头系假咯。睇下你系乜呀。睇下（丐时）系乜。"Output:{'头': False}
2020-02-28 12:09:01.580252To Show extra informationpython -m toisan_lang "你系嚿叉烧。你个头系假咯。睇下你系乜呀。睇下（丐时）系乜。" --show tree codeOutput:Parse Tree:
(START
  (begin_program (begin_scope))
  (block
    (statement (st_assign
      (var_list (var (variable_ref '你'))) '系'
      (exp_list (exp (dict_init '嚿叉烧'))))) '。'
    (statement (st_assign
      (var_list (var (variable_ref '你' '个' '头'))) '系'
      (exp_list (exp (constant (boolean '假')))))) '咯' '。'
    (statement (st_print '睇下'
      (var (variable_ref '你')) '系乜')) '呀' '。'
    (statement (st_print '睇下'
      (adjusted_exp '（'
        (exp (now '丐时')) '）') '系乜')) '。')
  (end_program (end_scope)))

Transpiled Python Code:
from datetime import datetime

def main():
    你 = dict()
    你['头'] = False
    print( 你 )
    print( ( datetime.now() ) )

if __name__ == '__main__':
    main()

{'头': False}
2020-03-03 22:26:13.549966Code Usagefromtoisan_langimportparseprogram="""..."""code,tree=parse(program)print(code)print(tree)exec(code)Developgit clone https://github.com/zehengl/toisan-lang.git
cd toisan-lang
python -m venv .venv
source .venv/bin/activate
pip install -r requirements-dev.txtTo run test, simplypytestHappy hacking!Creditslogo fromhttps://izihun.com/yishuzi/564471.html
Package: toishi
toishi日本語テキストのフォーマッター。
名称は、日本の伝統的な刃物の研ぎに使用される砥石(toishi)をモチーフにしています。開発背景日本語テキストをテキストエディタで修正しつつ、リポジトリでバージョン管理することがありました。バージョン管理にあたり、行ごとに改行が入っている方が履歴が追跡しやすくなります。日本語を書くのに夢中になり、不必要な改行が入ることもあります。しかし、それを手作業でチェックするのは煩雑です。コードフォーマッターのように自動で修正してほしい、という想いから作りました。コンセプト表現の柔軟性を削り、文章を型にはめることで、文章が持つ情報そのものに注目させる。フォーマットの自由度を落とすことで、利用者の表現に関する議論を削減する。手動でのフォーマット作業を省くことで、執筆者をより重要な作業に集中させる。インストールpip install toishi利用方法filename.txtに任意のファイル名を指定してください。toishi filename.txt機能(開発中)現在、開発中です。以下は追加予定の機能です。「。」の後に改行を自動で入れる。空行は連続しない。1行のみ。 複数行ある場合は、1行にする。ファイルの先頭の空行、最後の空行は削除する。チェックのみで修正を入れないオプション（CI用）ディレクトリを指定し、一括で処理する機能
Package: toJiraTable
Reads input from stdin and converts it to a JIRA table.The first line it reads is used as the table header, and all subsequent lines are made into the body. Supports shell-style splitting of fields (you can use quotes to “group” items with spaces into a single field).By default, it will split by whitespace, so each word becomes an item. The entire table, by default, will grow to the longest row, and fill in blanks where needed.If you need to split by an alternative thing, such as two-spaces (if columns are seperated by two-or-more spaces, and contain characters with spaces between) there are various --split-by options available, see usage below. See usage and examples for more information.Usage:Usage: toJiraTableConverts stdin to a JIRA table. If no arguments provided, it will use “shell-style” splittingso quoting strings with spaces makes them a single column, otherwise spaces/tabs/whateversplits. The first line passed in becomes the header, the remainder become the body.By default, everything will be matched to the longest row. Any missing columns in a row willbe filled by blank columns at the end. Use --no-stretch to disable this.Arguments:--no-stretch                -  Do not stretch each row to the longest row. See above.--split-header-by=X         -  Instead of using shell-style splitting, split by provided string for the header line--split-body-by=X           -  Instead of using shell-style splitting, split by the provided string for body lines--split-by=X                -  Split both header and body by the given string--split-keep-empty          -  By default, using the --split-by* will strip empty columns. The default behaviour isuseful, in example: if a script outputs strings which are not quoted, but has at least twospaces between each real column, using --split-body-by=’  ‘  will ensure that any place thatis separated by two or more spaces becomes a column. This option disables that feature.Example Usage:Simple split:[myuser]$ ( echo ‘Hostname “Free Space”’; cat hostnames.txt ) | toJiraTable||Hostname||Free Space|||host1|500G||wwwprod1|120G||wwwdev1|11G|More complicated split. In this case, we want the body to form columns any time there are at least two spaces:(dataset)[myuser]$ cat myDataName Size CommentGeorge Jetson  500G    The man from the futureJeeves  200M           Everyone’s favourite butlet(convert to JIRA table)[myuser]$ cat myData | ./toJiraTable --split-body-by=”  “||Name||Size||Comment|||George Jetson|500G|The man from the future||Jeeves|200M|Everyone’s favourite butlet|Tab-deliminated split:(dataset):[myuser]$ python -c “import sys; sys.stdout.write(repr(open(‘myData’, ‘rt’).read()).replace(’\\n’, ‘\n’) + ‘n’);”“NametSizetCommentGeorge Jetsont500GtThe man from the futureJeevest200MtEveryone’s favourite butlet“(convert to JIRA table)[myuser]$ cat myData | ./toJiraTable --split-by=’t’||Name||Size||Comment|||George Jetson|500G|The man from the future||Jeeves|200M|Everyone’s favourite butlet|
Package: tojson
CSV2JSON🚀 A CLI tool to export CSV files to JSON format.Requirements🐍python>=3.7⬇️ Installationpipinstalltojson⌨️ Usage➜ csv2json --help  # You can also use the alias `tojson` or `csvtojson`

usage: csv2json [-h] [-o {dict,list,series,split,records,index}] [-i INDENT]
                [-c]
                path [path ...]

positional arguments:
  path                  Path to a single file/directory or multiple
                        files/directories

options:
  -h, --help            show this help message and exit
  -o {dict,list,series,split,records,index}, --orient {dict,list,series,split,records,index}
                        The type of the values of the dictionary
  -i INDENT, --indent INDENT
                        JSON output indentation
  -c, --compress        Compress the output📕 Examplescsv2jsonfoo.csv# Orient as listcsv2jsonfoo.csv-olist# Compressed output and use zero indentation (minified)csv2jsonfoo.csv-c-i0
Package: to-json-schema
A simple package that convert dictionary or a list into json-schema using the OpenAPI standards.Installationto-json-schema can be installed viapip:$pipinstallto-json-schemaQuickstartYou can now convert your python dict into json-schema:fromto_json_schema.to_json_schemaimportSchemaBuilderdata={"country":"Pakistan","city":"Karachi","population":16051521,"is_capital":False,"key_qualities":["food","multi-cultural"],}schema_builder=SchemaBuilder()json_schema=schema_builder.to_json_schema(data)>>>{'type':'object','properties':{'country':{'type':'string'},'city':{'type':'string'},'population':{'type':'integer'},'is_capital':{'type':'boolean'},'key_qualities':{'type':'array','items':{'type':'string'}}}}You can also convert list of dict, string, integer into-json schemafromto_json_schema.to_json_schemaimportSchemaBuilderdata=[{"foo":12,"bar":14},{"moo":52,"car":641},{"doo":6,"tar":84}]schema_builder=SchemaBuilder()json_schema=schema_builder.to_json_schema(data)>>>{'type':'array','items':{'type':'object','properties':{'foo':{'type':'integer'},'bar':{'type':'integer'},'moo':{'type':'integer'},'car':{'type':'integer'},'doo':{'type':'integer'},'tar':{'type':'integer'}}}}
Package: tok
## tok[![PyPI](https://img.shields.io/pypi/v/tok.svg?style=flat-square)](https://pypi.python.org/pypi/tok/)
[![PyPI](https://img.shields.io/pypi/pyversions/tok.svg?style=flat-square)](https://pypi.python.org/pypi/tok/)Fast and most complete/customizable tokenizer in Python.It is roughly 25x faster than spacy’s and nltk’s regex based tokenizers.Using the aho-corasick algorithm makes it a novelty and allows it to be both explainable and fast in how it will split.The heavy lifting is done by [textsearch](https://github.com/kootenpv/textsearch) and [pyahocorasick](https://github.com/WojciechMula/pyahocorasick), allowing this to be written in only ~200 lines of code.Contrary to regex-based approaches, it will go over each character in a text only once. Read [below](#how-it-works) about how this works.### Installationpip install tok### UsageBy default it handles contractions, http, (float) numbers and currencies.`python from tok import word_tokenizeword_tokenize("Iwouldn't dothat....wouldyou?")['I', 'would', 'not', 'do', 'that','...','would', 'you','?']`Or configure it yourself:`python from tok import Tokenizer tokenizer =Tokenizer(protected_words=["some.thing"])# still using the defaultstokenizer.word_tokenize("Iwant to protect some.thing") ['I', 'want', 'to', 'protect', 'some.thing'] `Split by sentences:`python from tok import sent_tokenizesent_tokenize("Iwouldn't dothat....wouldyou?")[['I','would', 'not', 'do', 'that','...'],['would', 'you','?']]`for more options check the documentation of theTokenizer.### Further customizationGiven:`python from tok import Tokenizer t =Tokenizer(protected_words=["some.thing"])# still using the defaults `You can add your own ideas to the tokenizer by using:t.keep(x, reason): Whenever it finds x, it will not add whitespace. Prevents direct tokenization.t.split(x, reason): Whenever it finds x, it will surround it by whitespace, thus creating a token.t.drop(x, reason): Whenever it finds x, it will remove it but add a split.t.strip(x, reason): Whenever it finds x, it will remove it without splitting.`pythont.drop("bla","bla is not needed")t.word_tokenize("Pleaseremove bla, thank you") ['Please', 'remove',',','thank', 'you'] `### ExplainableExplain what happened:`pythont.explain("bla")[{'from':'bla', 'to': ' ', 'explanation': 'bla is notneeded'}]`See everything in there (will help you understand how it works):`python t.explain_dict `### How it worksIt will always only keep the longest match. By introducing a space in your tokens, it will make it be split.If you consider how the tokenization of.works, see here:When it finds a ` A.` it will make it ` A.` (single letter abbreviations)When it finds a.0it will make it.0(numbers)When it finds a., it will make it ` . ` (thus making a split)If you want to make sure something including a dot stays, you can use for example:t.keep(“cool.”)### ContributingIt would be greatly appreciated if you want to contribute to this library.It would also be great to add [contractions](https://github.com/kootenpv/contractions) for other languages.
Package: tok715
tok715开发代号: TOK715依赖ffmpeg, 使用ffmpeg捕捉麦克风数据流poetry, 使用poetry管理 Python 依赖组件tok715-voicerecog: 语音识别模块功能使用ffmpeg捕捉麦克风数据流传输数据流到阿里云语音识别服务使用redis发布识别结果到自然语言输入主题编码格式阿里云识别服务要求编码格式为pcm_s16le，采样率为16000，单声道。tok715-vectorstor: 存储和预处理模块Redis PUB/SUB 主题自然语言输入tok715:nl-input:USER_GROUP:USER_ID{"ts":1698240603702,"text":"坐在驶进乡间小路的摇晃公交车里，他回想起了过去唯一次和祖母交谈。","user":{"id":"owner","name":"主人","group":"owner"}}配置文件# tok715.yml# 用户身份# 使用模块：voicerecoguser:id:ownergroup:ownername:主人# redis 配置# 更多参数，翻阅 https://redis-py.readthedocs.io/en/stable/connections.html# 使用模块: voicerecogredis:host:'127.0.0.1'port:6379# 阿里云 服务配置# 使用模块: voicerecogaliyun:# 阿里云语音识别nls:endpoint:wss://nls-gateway.cn-shanghai.aliyuncs.com/ws/v1app_key:xxxxxaccess_key_id:xxxxxaccess_key_secret:xxxxx许可证TOK715 Developers, MIT License
Package: tokamesh
TokameshTokamesh is a Python package which provides tools for constructing meshes
and geometry matrices used in tomographic inversion problems in toroidal
fusion-energy devices such asMAST-U.FeaturesAdvanced geometry matrix calculationTokamesh constructs geometry matrices using barycentric linear-interpolation rather
than the typical zeroth-order interpolation. This allows for accurate tomographic
inversions with a significantly lower number of basis functions.Tomography-optimised mesh constructionTokamesh provides tools to create meshes that are optimised for tomography problems,
e.g. local-refinement of triangles to increase mesh density in areas where it is
needed without greatly increasing the size of the mesh.Jupyter notebook demosJupyter notebooks are available which demonstratemesh constructionandgeometry matrix calculation.InstallationTokamesh is available fromPyPI,
so can be easily installed usingpipas follows:pipinstalltokameshDocumentationThe package documentation is available attokamesh.readthedocs.io.
Package: tokapi-client
The simple http client forTokApitiktok mobile APIInstallationpipinstalltokapi-clientUsage# For RapidAPIapi=TokApi('YOUR_RAPID_API_KEY')# For api.tokapi.onlineapi=TokApi('YOUR_API_KEY_FROM_DASHBOARD',base_url="https://api.tokapi.online")# Let's find some users by search query with paginationkeyword='nike'offset=0foriinrange(0,3):result=api.search_user_by_keyword(keyword,offset=offset)data=result.json()foruserindata['user_list']:info=user['user_info']print('Nickname: {}, region: {}'.format(info['nickname'],info['region']))offset=data['cursor']ExamplesYou can find more complex usage examples inexamplesfolderLegalThis code is in no way affiliated with, authorized, maintained, sponsored or endorsed by TikTok
or any of its affiliates or subsidiaries. This is an independent and unofficial API. Use at your own risk.
Package: tokeks
tokeksA zero-dependency Python library for getting the Kubernetes token of a AWS EKS cluster.NoAWS CLI, third-party client or library (boto3, botocore, etc.) required.InstallationpipinstalltokeksUsage prerequisitesAssign your credentials to environment variables:exportAWS_ACCESS_KEY_ID="..."exportAWS_SECRET_ACCESS_KEY="..."Library usagefromtokeksimportTokekstokeks=Tokeks(access_key_id=os.environ['AWS_ACCESS_KEY_ID'],secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],region_id='YOUR_REGION_ID',cluster_id='YOUR_CLUSTER_ID')token=tokeks.get_token()CLI usageGo to project directoryRun script:TOKEN=$(./getekstokenus-west-1cluster-1)Verify Kubernetes API access:APISERVER=...
curl-XGET$APISERVER/api--header"Authorization: Bearer$TOKEN"--insecureIf the response is something like:{"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCIDR":"0.0.0.0/0","serverAddress":"..."}]}(and does not contain "Unauthorized"), you're good to go!Inspired by:API Authorization from Outside a Clustereks-token
Package: tokelor
tokelorVisualize Python token stream produced by tokenize module.ExampleUsage# tokelor --help
Usage: tokelor [OPTIONS] SOURCE

  Visualize Python token stream produced by tokenize module.

Options:
  --nl / --no-nl      Display newline tokens.
  --bare / --no-bare  Replace bold text with underlinings.
  --help              Show this message and exit.Installation# pip install tokelorGetting started with development# git clone git@github.com:lensvol/tokelor.git# poetry install --developLicenseThis project is licensed under the MIT License - see theLICENSEfile for detailsAuthorsKirill Borisov(lensvol@gmail.com)
Package: tokema
tokematokema -tokenmatching parser.It is a pure python, zero-dependency library for building parsers.
tokema algorithm based on Tomita's GLR* parser.It helps to create noise-skipping grammar-based parsers at runtime and apply them to various task like
entity extraction and token matching.tokema algorithm is the extended version on Noise-skipping GLR* parser by Lavie Alon and Masaru Tomita:Lavie, Alon, and Masaru Tomita.
"10. GLR*-AN EFFICIENT NOISE-SKIPPING PARSING ALGORITHM FOR CONTEXT-FREE GRAMMARS."
Recent Advances in Parsing Technology 1 (1996): 183.tokema parsing works with arbitrary tokens - chars, words, structures... you can write a parser for
token stream produced by the tokenizer of your choice.GLRIn general, tokema - is a GLR-based (Generalized LR) context-free parser generator which basically means
that parser produced by tokema tries to find most suitable parses (might be multiple) given deterministic grammar.
GLR-based algorithms evaluate multiple parser states at the same time allowing to parse using incomplete and poorly structured grammars.Noise-skippingThe term "noise skipping" means that parser can skip tokens it doesn't understand which is required to solve
natural language tasks.ExtensionsInstead of direct token matching (or dictionary based) like in most traditional parsers tokema
replaces terminals and non-terminals in the grammar with queries and resolvers -
functions that matches the input token with the given query.By creating these functions you can construct token queries and resolvers of your choice integrating lots of cool features into your parser.
For example you can create a custom resolver matches stemmed or lemmatized text tokens.
Or create a resolver that uses custom dictionaries or even integrate a third party search backend like lucene.Installationpip install tokemaUsageBasic usage:fromtokemaimport*# Define grammar# NOTE: you can also define it using python classes only or create you own grammar syntaxrules=parse_rules_from_string("""ROOT = <EXPR>EXPR ={float}+{float}""")# Parsing table constructiontable=build_text_parsing_table(rules)# Input tokens# NOTE: In real application you should use some real tokenizer (like nltk)tokens='this will be ignored 3.1415 and + this 4e-10'.split()# Actual parsing using token stream and parsing table# There may be multiple parses (not in this oversimplified scenario) so parse returns a listresults=parse(tokens,table)forresultinresults:# Here you can access the parsed AST (abstract syntax tree)print(result)# ROOT(EXPR(3.1415, +, 4e-10))# EXPR is the first child of ROOTexpr=result[0]# Convert each value to a float and make sure that result is actually correctassertfloat(expr[0].value)+float(expr[2].value)==3.1415+4e-10For more usage scenarios see examples folder
Package: tokenauth
=====================================================================Tokenauth=====================================================================Tokenauth developed for django rest token authentication. It enables you to define expiration time and token prefix. Tokens generate with random data and it use SHA-256 algorithm.Installation============on Pypi::pip install tokenauthon Github::git clone git@github.com:ahmetkotan/tokenauth.gitcd tokenauthpython setup.py installSettings============Added to `INSTALLED_APPS`.::INSTALLED_APPS = [...'tokenauth',...]Added to `urls.py`.::urlpatterns = [...url(r'^<your-login-url>/', include('tokenauth.urls')),...]Definition in `settings.py`::TOKEN_EXPIRATION_TIME = 60 * 60 * 24 * 3 # Default 3 daysTOKEN_PREFIX = "Bearer"TOKEN_REFRESH = TrueUsage=====Definition in `settings.py`::# Rest FrameworkREST_FRAMEWORK = {...'DEFAULT_AUTHENTICATION_CLASSES': ('tokenauth.auth.TokenAuthentication',),...}Or in `views.py`::from tokenauth.auth import TokenAuthenticationclass SimpleView(ModelViewset):authentication_classes = (TokenAuthentication, )Created token and login:::curl -X POST -H "Content-Type: application/json" -d '{"username": "<username>", "password":"<password>"}' <your-django-url>/<your-login-url>/tokens/Refresh token:::curl -X PUT -H "Content-Type: application/json" -d '{"key": "<your-valid-token>"}' <your-django-url>/<your-login-url>/tokens/Deleted token and logout:::curl -X DELETE -H "Content-Type: application/json" -H "Authorization: <your-token>" <your-django-url>/<your-login-url>/tokens/
Package: token-auth-cli
token-auth-cliDescriptionSimple tool for testing token authentication during development.Installationpip install token-auth-cliorgit clone https://github.com/0djentd/token-auth-cli.git
cd token-auth-cli
python -m pip install .How to useUsage: token-auth-cli [OPTIONS] COMMAND [ARGS]...

  token-auth-cli

  Simple tool for testing token authentication during development.

Options:
  --verbose / --no-verbose    Show additional information
  --debug / --no-debug        Show debug information
  --confirm-settings BOOLEAN  Confirm settings before trying to get token.
  --show-settings BOOLEAN     Show settings before trying to get token.
  --api TEXT                  API url.
  --api-get-token TEXT        API url to use when trying to get token.
  --api-get TEXT              API url to check if token valid.
  --config FILE               Config file.
  --store BOOLEAN             Store users/tokens.
  --help                      Show this message and exit.

Commands:
  init     Create config file and users/tokens storage.
  list     List stored users/tokens.
  login    Get token and store it.
  relogin  Try to use stored token for authentication.
  remove   Remove stored user/token.
Package: tokenbucket
# Token bucketA simple rate limiting algorithm. See
[Wikipedia](http://en.wikipedia.org/wiki/Token_bucket) for a more lengthy
discussion of how it works and where it is applicable. In essence, it is a
constant rate limiting with temporal pooling. This allows for bursts of
activity.## Memcached sharingEnables sharing across different processes and servers, useful for rate
limiting a shared resource (like a Web API.)from tokenbucket import SharedTokenBuckettb = SharedTokenBucket(rate=1.0, maximum=5)tb.use_memcached_key(‘my_shared_token_bucket’)
# or
tb.use_memcached_key(‘my_shared_token_bucket’, client=mc)