Saravanan666/pypi-org huggingface dataset:
Package: pangeamt-tea
TEA - Translation Engine ArchitectA command line tool to create translation engine.InstallFirst installpipxthen (x being your python version):pipx install pangeamt-teaUsageStep 1: Create a new projecttea new --customer customer --src_lang es --tgt_lang en --flavor automotion --version 2This command will create the project directory structure:├── customer_es_en_automotion_2
│   ├── config.yml
│   └── dataThen enter in the directorycd customer_es_en_automotion_2Step 2: ConfigurationTokenizerA tokenizer can be applied to source and targettea config tokenizer --src mecab  --tgt mosesTo list all available tokenizer:tea config tokenizer --helpif you would not like to use tokenizers you can run:tea config tokenizer -s none -t noneTruecasertea config truecaser --src --tgtif you would not like to use truecaser you can run:tea config tokenizerBPE / SentencePieceFor joint BPE:tea config bpe -jFor not joint BPE:tea bpe -s -tFor using sentencepiece:tea config bpe --sentencepieceand options --model_type TEXT (unigram) --vocab_size INTEGER (8000) if you would like to modify them from defaultProcessorstea config processors -s "{processors}"being processors a list of preprocesses and postprocesses.To list all available processors:tea config processors --listIn order to test the processors that will be applied you can run this script in the main TEA project directory:debug_normalizers.py <config_file> <src_test> <tgt_test>being config_file the yaml config and src_test and tgt_test the segments to test for source and target text.Preparetea config prepare --shard_size 100000 --src_seq_length 400 --tgt_seq_length 400Translation modeltea config translation-model -n onmtStep 3:Copy some multilingual ressources (.tmx, bilingual files, .af ) into the 'data' directoryStep 4: RunCreate workflowtea worflow newClean the data passing the normalizers and validators:tea workflow clean -n {clean_th} -dbeing clean_th the number of threads.Preprocess the data (split data in train, dev or test, tokenization, BPE):tea workflow prepare -n {prepare_th} -s 3being prepare_th the number of threads.Training modeltea workflow train --gpu 0if you do not want to use gpu do not use this parameter.Evaluate modeltea workflow eval --step {step} --src file.src --ref file.tgt --log file.log --out file.out --gpu 0ResetFirst of all you may check the current status of the workflow using:tea workflow statusThen you can reset your worflow at any step  (clean, prepare, train, eval) using:tea worflow reset -s {step_name}Or if you want to make a full reset of the workflow use:tea workflow resetIf you need some help on how to use reset command:tea workflow reset --help
Package: pan-genomics-download
pan_genomics_downloadpgd, short for pan_genomics_download, is a convenient command line tool to download all files from pan.genomics.cn using a shared link.InstallationFrom pypi$ pip install pan-genomcis-downloadFrom github$ git clone https://github.com/JaylanLiu/pan-genomcis-download
$ cd pan-genomcis-download
$ python setup.py installUsage$ pgd -h
usage: pgd [-h] -u URL [-o TARGET_PATH] [-s SUFFIX] [-f]

pan-genomics_download

optional arguments:
  -h, --help            show this help message and exit
  -u URL, --url URL     shared link
  -o TARGET_PATH, --target_path TARGET_PATH
                        target path to store files
  -s SUFFIX, --suffix SUFFIX
                        allowed file suffix
  -f, --flat            ignore the directory hierarchy and download all files to target path directlyExample$pgd -u https://pan.genomics.cn/ucdisk/s/Qzaxxxxxxx
2021-06-18 14:51:02,091 - pan_genomics_download.py[line:38] - INFO: get files list successfully
2021-06-18 14:51:03,042 - pan_genomics_download.py[line:84] - INFO: ./生信分析文件/2021年6月/f300007033_202106151126.zip download successfully
2021-06-18 14:51:03,801 - pan_genomics_download.py[line:84] - INFO: ./生信分析文件/2021年6月/F300007060_202106091332.zip download successfully
2021-06-18 14:51:06,400 - pan_genomics_download.py[line:84] - INFO: ./生信分析文件/2021年5月/F300007070_202105240906分析文件.zip download successfully
Package: pangeobench
BenchmarkingBenchmarking & Scaling Studies of the Pangeo PlatformBenchmarkingCreating an EnvironmentBenchmark ConfigurationRunning the BenchmarksBenchmark ResultsVisualizationCreating an EnvironmentTo run the benchmarks, it's recommended to create a dedicated conda environment by running:condaenvcreate-f./binder/environment.ymlThis will create a conda environment namedpangeo-benchwith all of the required packages.You can activate the environment with:condaactivatepangeo-benchand then run the post build script:./binder/postBuildBenchmark ConfigurationThebenchmark-configsdirectory contains YAML files that are used to run benchmarks on different machines. So far, the following HPC systems' configs are provided:$tree./benchmark-configs/
benchmark-configs/
├──cheyenne.yaml
└──hal.yaml
└──wrangler.yamlIn case you are interested in running the benchmarks on another system, you will need to create a new YAML file for your system with the right configurations. See the existing config files for reference.Running the Benchmarksfrom command lineTo run the benchmarks, a command utilitypangeobenchis provided in this repository.
To use it, you simply need to specify the location of the benchmark configuration file. For example:./pangeobenchbenchmark-configs/cheyenne.yaml$./pangeobench--help
Usage:pangeobench[OPTIONS]CONFIG_FILE

Options:--helpShowthismessageandexit.Running the Benchmarksfrom jupyter notebook.To run the benchmarks from jupyter notebook, install 'pangeo-bench' kernel to your jupyter notebook enviroment, then start run.ipynb notebook.  You will need to specify the configuration file as described above in your notebook.To install your 'pangeo-bench' kernel to your jupyter notebook enviroment you'll need to connect a terminal of your HPC enviroment and run following command.source activate pangeo-bench
ipython kernel install --user --name pangeo-benchBefore starting your jupyternotebook, you can verify that if your kernel is well installed or not by follwing commandjupyter kernelspec listBenchmark ResultsBenchmark results are persisted in theresultsdirectory by default. The exact location of the benchmark results depends on the machine name (specified in the config file) and the date on which the benchmarks were run. For instance, if the benchmarks were run on Cheyenne supercomputer on 2019-09-07, the results would be saved in:results/cheyenne/2019-09-07/directory. The file name follows this template:compute_study_YYYY-MM-DD_HH-MM-SS.csvVisualizationVisualisation can be done using jupyter notebooks placed in analysis directories.
Package: pangeo-forge
pangeo-forgepangeo-forgeis an open-source tool designed to aid the extraction, transformation, and loading of datasets. The goal of pangeo-forge is to make it easy to extract datasets from traditional data repositories and deposit them into cloud object storage in analysis-ready, cloud-optimized format.pangeo-forge is inspired byconda-forge, a community-led collection of recipes for buildingCondapackages. We hope that pangeo-forge can play the same role for datasets.DocumentationMore can be learned about pangeo-forge, its progress, and related subprojects in itsofficial documentation.Contributingpangeo-forge is still early in development - there are several ways to contribute:Create a recipefor a dataset you are interested inOpen an issue or pull request here or in any of the related subprojects (pangeo-smithy,staged-recipes)Check out the projectroadmapGet in touchDiscussions on pangeo-forge are generally hosted biweekly on Mondays at 7pm UTC viaWhereby. More details on the scheduling of these meetings can be foundhere.LicenseThis project is licensed under theApache License, Version 2.0.
Package: pangeo-forge-cmr
pangeo-forge-cmrSmall library that integrates NASA'sCommon Metadata Repository(CMR) withpangeo-forge-recipes.
The goal is to help make pangeo-forge recipes that use CMR for getting raw data.InstallationYou can install this from PyPI withpip install pangeo-forge-cmr.Examplefrompangeo_forge_recipes.recipesimportXarrayZarrRecipefrompangeo_forge_cmrimportfiles_from_cmr# Get the GPM IMERG Late Precipitation Daily datashortname='GPM_3IMERGDL'recipe=XarrayZarrRecipe(# We are making Zarr, could be something else toofiles_from_cmr(# Provide a list of files by querying CMRshortname,nitems_per_file=1,concat_dim='time',# Describe how the dataset is chunked),inputs_per_chunk=12,)CreditMost of the CMR code was developed byBrianna Pagáninthis PR
Package: pangeo-forge-esgf
pangeo-forge-esgfUsing queries to the ESGF API to generate urls and keyword arguments for receipe generation in pangeo-forgeParsing a list of instance ids using wildcardsPangeo forge recipes require the user to provide exact instance_id's for the datasets they want to be processed. Discovering these with theweb searchcan become cumbersome, especially when dealing with a large number of members/models etc.pangeo-forge-esgfprovides some functions to query the ESGF API based on instance_id values with wildcards.For example if you want to find all the zonal (uo) and meridonal (vo) velocities available for thelgmexperiment of PMIP, you can do:frompangeo_forge_esgf.parsingimportparse_instance_idsparse_iids=["CMIP6.PMIP.*.*.lgm.*.*.uo.*.*","CMIP6.PMIP.*.*.lgm.*.*.vo.*.*",]iids=[]forpiidinparse_iids:iids.extend(parse_instance_ids(piid))iidsand you will get:['CMIP6.PMIP.MIROC.MIROC-ES2L.lgm.r1i1p1f2.Omon.uo.gn.v20191002',
 'CMIP6.PMIP.AWI.AWI-ESM-1-1-LR.lgm.r1i1p1f1.Odec.uo.gn.v20200212',
 'CMIP6.PMIP.AWI.AWI-ESM-1-1-LR.lgm.r1i1p1f1.Omon.uo.gn.v20200212',
 'CMIP6.PMIP.MIROC.MIROC-ES2L.lgm.r1i1p1f2.Omon.uo.gr1.v20200911',
 'CMIP6.PMIP.MPI-M.MPI-ESM1-2-LR.lgm.r1i1p1f1.Omon.uo.gn.v20200909',
 'CMIP6.PMIP.AWI.AWI-ESM-1-1-LR.lgm.r1i1p1f1.Omon.vo.gn.v20200212',
 'CMIP6.PMIP.MIROC.MIROC-ES2L.lgm.r1i1p1f2.Omon.vo.gn.v20191002',
 'CMIP6.PMIP.AWI.AWI-ESM-1-1-LR.lgm.r1i1p1f1.Odec.vo.gn.v20200212',
 'CMIP6.PMIP.MIROC.MIROC-ES2L.lgm.r1i1p1f2.Omon.vo.gr1.v20200911',
 'CMIP6.PMIP.MPI-M.MPI-ESM1-2-LR.lgm.r1i1p1f1.Omon.vo.gn.v20190710']Eventually I hope I can leverage this functionality to handle user requests in PRs that add wildcard instance_ids, but for now this might be helpful to manually construct lists of instance_ids to submit to a pangeo-forge feedstock.Generating PGF recipe input (urls) from instance_idsfrompangeo_forge_esgfimportget_urls_from_esgfiids=['CMIP6.CMIP.CSIRO-ARCCSS.ACCESS-CM2.historical.r1i1p1f1.SImon.sifb.gn.v20200817']url_dict=awaitget_urls_from_esgf(iids)url_dict['CMIP6.CMIP.CSIRO-ARCCSS.ACCESS-CM2.historical.r1i1p1f1.SImon.sifb.gn.v20200817']gives100%|██████████| 5/5 [00:01<00:00,  4.98it/s]
Processing responses
Processing responses: Expected files per iid
Processing responses: Check for missing iids
Processing responses: Flatten results
Processing responses: Group results
Find responsive urls
100%|██████████| 1/1 [00:00<00:00,  3.25it/s]
['https://esgf-data1.llnl.gov/thredds/fileServer/css03_data/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/SImon/sifb/gn/v20200817/sifb_SImon_ACCESS-CM2_historical_r1i1p1f1_gn_185001-201412.nc']or if you want to see detaile debugging statementsfrompangeo_forge_esgfimportget_urls_from_esgf,setup_loggingsetup_logging('DEBUG')iids=['CMIP6.CMIP.CSIRO-ARCCSS.ACCESS-CM2.historical.r1i1p1f1.SImon.sifb.gn.v20200817']url_dict=awaitget_urls_from_esgf(iids)url_dict['CMIP6.CMIP.CSIRO-ARCCSS.ACCESS-CM2.historical.r1i1p1f1.SImon.sifb.gn.v20200817']
Package: pangeo-forge-recipes
pangeo-forge-recipespangeo-forgeis an open-source tool designed to aid the extraction, transformation, and loading of datasets. The goal of pangeo-forge is to make it easy to extract datasets from traditional data repositories and deposit them into cloud object storage in analysis-ready, cloud-optimized format.pangeo-forge is inspired byconda-forge, a community-led collection of recipes for buildingCondapackages. We hope that pangeo-forge can play the same role for datasets.DocumentationMore can be learned about pangeo-forge, its progress, and related subprojects in itsofficial documentation.Contributingpangeo-forge is still early in development - there are several ways to contribute:Create a recipefor a dataset you are interested inOpen an issue or pull request here or in any of the related subprojects (pangeo-smithy,staged-recipes)Check out the projectroadmapGet in touchDiscussions on Pangeo Forge are generally hosted biweekly on Mondays at 2pm ET viaWhereby.
Calendar linkhere.
We aim to announce cancellations onthis discourse thread.LicenseThis project is licensed under theApache License, Version 2.0.
Package: pangeo-sphinx-book-theme
Pangeo Sphinx Book ThemeThis is a theme forSphinxthat inherits from theSphinx Book Theme.It is used for thePangeo Forge documentationand related projects.Build the documentationThis site comes bundled with a small documentation site to make it easy to preview what the theme looks like.The easiest way to build the documentation is to usthe nox command line tool.
This is kind-of like aMakefilethat is bundled with an isolated local environment.To build the documentation withnox, run this command:nox -s docsTo build the documentation with a live server that auto-reloads when you make changes todocs/, run this command:nox -s docs-liveConfiguration fornoxcan be found innoxfile.py.
Package: pangeo-xesmf
xESMF is a Python package forregridding.
It isPowerful: It usesESMF/ESMPyas backend and can regrid betweengeneral curvilinear gridswith allESMF regridding algorithms,
such asbilinear,conservativeandnearest neighbour.Easy-to-use: It abstracts away ESMF’s complicated infrastructure
and provides a simple, high-level API, compatible withxarrayas well as basic numpy arrays.Fast: It is faster than ESMPy’s original Fortran regridding engine in serial case, and also supportsdaskforout-of-core, parallel computation.Please seeonline documentation, orplay with example notebooks on Binder.For new users, I also recommend readingHow to ask for helpandHow to support xESMF.
Package: panglery
pangleryis a library for writing hooks for events in python.Here’s a basic example of usage:import panglery
p = panglery.Pangler()

@p.subscribe(event='example', needs=['spam'])
def example_hook(p, spam):
    print spam

p.trigger(event='example', spam='eggs')
# prints 'eggs'And a little bit more involved:@p.subscribe(needs=['spam'], returns=['spam'])
def modify_spam_hook(p, spam):
    spam = spam + ' spam'
    return {'spam': spam}

p.trigger(event='example', spam='eggs')
# prints 'eggs spam'Hooks can also add parameters to an event which then trigger other hooks:p = panglery.Pangler()

@p.subscribe(needs=['spam'], returns=['eggs'])
def make_eggs_hook(p, spam):
    eggs = spam + ' eggs'
    return {'eggs': eggs}

@p.subscribe(event='example', needs=['eggs']):
def eggs_hook(p, eggs):
    print eggs

p.trigger(event='example', spam='eggs')
# prints 'eggs eggs'PanglerAggregates can be used to aggregate together multiple Panglers across
all superclasses:class ExampleBase(object):
    p = panglery.PanglerAggregate('hooks')
    hooks = panglery.Pangler()

    @hooks.subscribe(event='example')
    def example_hook_base(self, p):
        print 'spam'

class ExampleDerived(ExampleBase):
    hooks = panglery.Pangler()

    @hooks.subscribe(event='example')
    def example_hook_derived(self, p):
        print 'eggs'

inst = ExampleDerived()
inst.p().trigger(event='example')
# prints 'spam' and 'eggs' in some order.
Package: pangloss
# panglossProvides support for interlinear glosses with Markdown example lists.## ExampleThe following code snippet demonstrates the most important features ofpangloss:```As you can see in the following examples, pangloss is really easy to use:(@) Jorge  llama             a  Maria.George calls-3s.PRES.IND to Maria'George calls Maria.'(@) Aussi, vous pouvez          avoir    de multiples   exemples.also   you  can-2p.PRES.IND have.INF of multiple-PL example-PL'You can also have multiple examples.' {#ex:french}You can even refer to examples, as in @ex:french.```Each example consists of three lines: an original, a word-by-word analysis, andan overall translation. Placing `{#ex:...}` after the translation lineintroduces a new label, which can then be referred to with the `@ex:...`syntax as in [pandoc-crossref](https://github.com/lierdakil/pandoc-crossref).Similar customization of labels and more advanced references are coming soon.## InstallationInstall with:```pip install -U pangloss```Use with:```pandoc in.md -F pangloss -o out.{pdf,html}```
Package: pango-aliasor
Python convenience library to translate between aliased and unaliased Pango lineagesUseful for:constructing a tree of Pango lineagessemantic sorting of Pango lineages...Usagefrompango_aliasor.aliasorimportAliasor# Initalize aliasor (only needs to be done once)# If no alias_key.json is passed, downloads the latest version from githubaliasor=Aliasor()# To use custom alias_key.json, pass the relative path to the file# aliasor = Aliasor('alias_key.json')# Go from aliased lineage to unaliased lineagealiasor.uncompress("BA.5")# 'B.1.1.529.5'aliasor.uncompress("BE.5")# 'B.1.1.529.5.3.1.5'aliasor.uncompress("XA")# 'XA'# Go from unaliased lineage to aliased lineagealiasor.compress("B.1.1.529.3.1")# 'BA.3.1'# Find parent lineagealiasor.parent("BQ.1")# 'BE.1.1.1'aliasor.partial_compress("B.1.1.529.3.1",up_to=1)# 'BA.3.1'aliasor.partial_compress("B.1.1.529.3.1.1.2",up_to=1)# 'BA.3.1.1.2'aliasor.partial_compress("B.1.1.529.3.1",accepted_aliases=["AY"])# 'B.1.1.529.3.1'aliasor.partial_compress("B.1.617.2",accepted_aliases=["AY"])# 'AY.2'aliasor.partial_compress('B.1.1.529.2.75.1.2',up_to=4,accepted_aliases={"BA"})=='BL.2'Seetestsfor more examples.Installationpipinstallpango_aliasorTestingRunpytestfrom the project root to run all tests.ReleaseBump version insetup.cfgRelease usinggh release createBuild usingpython3 -m buildPublish to Pypi usingtwine upload dist/pango_aliasor*
Package: pangocffi
pangocffipangocffi is aCFFI-based set of Python bindings forpango.pangocffi on its own is not that useful, since it depends on a PangoFontMap
being declared against the PangoContext.
PangoFontMap instances can easily be retrieved from libraries such as
PangoCairo, PangoXft, PangoFT2, and PangoWin32 (See gnome's documentation'Rendering with Pango'for a list of rendering engines).Seepangocairocffifor bindings that allow you to render pango objects with
cairo.The bindings are currently not fully implemented. Feel free to make a pull
request to contribute!Installation and usageSee'Overview'for information on how to install the necessary libraries.See'Python API Reference'for additional information on all the objects.ContributingIf you would like to contribute to this project, either by leaving feedback or
submitting a pull request, please read 'CONTRIBUTING.md'.
Package: pangolier
Pangolierprometheus query builderinstallpip install pangolierusagesimple caseFor a metric with filters:from pangolier.metrics import Metric

print(Metric('http_requests_total').filter(
    job='prometheus',
    group='canary'
).to_str())output:http_requests_total{job="prometheus", group="canary"}pretty outputAddpretty=Trueinto_strfor better readability:from pangolier.metrics import Metric

print(Metric('http_requests_total').filter(
    job='prometheus',
    group='canary'
).to_str(pretty=True))output:http_requests_total{
    job="prometheus",
    group="canary"
}I will always usepretty=Truein rest of this document.functionsPrometheus functions can be built by name. For example:from pangolier.functions import function

abs = function('abs')
print(abs(Metric('http_requests_total')).to_str(pretty=True))output:abs(
    http_requests_total
)range_functionshould be used for functions accept arange-vector.from pangolier.functions import range_function

rate = range_function('rate')
print(rate(Metric('http_requests_total'), timespan='5m').to_str(pretty=True))output:rate(
    http_requests_total[5m]
)aggregation_operatorshoule be used for aggregation operators:from pangolier.functions import aggregation_operator

sum_ = aggregation_operator('sum')
print(sum_(
    Metric('http_requests_total'),
    by=['job', 'group'],
).to_str(pretty=True))output:sum by(
    job, group
)(
    http_requests_total
)combine them all together:histogram_quantile = function('histogram_quantile')
rate = range_function('rate')
sum_ = aggregation_operator('sum')

print(histogram_quantile(
    0.9,
    sum_(
        rate(
            Metric('http_request_duration_seconds_bucket'),
            timespan='5m',
        ),
        by=['le']
    )
).to_str(pretty=True))output:histogram_quantile(
    0.9,
    sum by(
        le
    )(
        rate(
            http_request_duration_seconds_bucket[5m]
        )
    )
)binary operatorssupport following binary operators:+(addition)-(subtraction)*(multiplication)/(division)%(modulo)^(power/exponentiation)For example, divide one metric with another:from pangolier.metrics import Metric
from pangolier.functions import range_function

rate = range_function('rate')
print((
    rate(
        Metric('foo').filter(
            group='canary'
        ),
        timespan='5m'
    ) / rate(
        Metric('bar').filter(
            group='canary'
        ),
        timespan='5m'
    )
).to_str(pretty=True))output:rate(
    foo{
        group="canary"
    }[5m]
) / rate(
    bar{
        group="canary"
    }[5m]
)For operation with modifier:from pangolier.metrics import Metric, BinOp, GroupLeft

print(BinOp(
    '*',
    Metric('foo'),
    Metric('bar'),
    on=['interface', 'job'],
    group=GroupLeft('node', 'resource'),
).to_str(pretty=True))output:foo * on(
    interface, job
) group_left(
    node, resource
) barabout namePangolier
Package: pangpuriye
pangpuriyeLet's make AI easy.FeaturesOCR- EasyOCR + Tesseract + Levenshtein,example.RequirementsBefore start we require 2-main libraries below :poppler- it used inpdf2image,how-to-installin each OS.tesseract- specially in MacOS, we need to install multi-language viabrewcommand :$brewinstalltesseract-langUsageWe requires library below inpangpuriye.aipackage and our example code, you should install the library below first  :pdf2imagepython-Levenshteineasyocropencv-pythontorchvisiondetectomatplotlibnumpyOr$pipinstallpdf2imagepython-Levenshteineasyocropencv-pythontorchvisiondetectomatplotlibnumpyAfter that let's installpangpuriye:$pipinstallpangpuriyeEnjoy!!!Issuespython levenshtein issue on window:https://stackoverflow.com/questions/13200330/how-to-install-python-levenshtein-on-windowspoppler usage in MacOS version lower than 10.15:Warning:YouareusingmacOS10.13.
We(andApple)donotprovidesupportforthisoldversion.
Youwillencounterbuildfailureswithsomeformulae.
PleasecreatepullrequestsinsteadofaskingforhelponHomebrew'sGitHub,
Twitteroranyotherofficialchannels.Youareresponsibleforresolving
anyissuesyouexperiencewhileyouarerunningthis
oldversion.

Error:poppler:nobottleavailable!
Youcantrytoinstallfromsourcewith:

brewinstall--build-from-sourcepopplerFAQWhy not pack the required liraries above into this package?Because of some libraries above rely on host's environment, examplepdf2imagerequirespopplerbut you need to findpopplerpackage that match with your host. So we did not pack the library above into our package.LicenseMIT