Saravanan666/pypi-org huggingface dataset:
Package: piperoni
PiperoniPiperoni is a lightweight ETL framework for any data type, which allows you to make, track, and visualize atomic data transformations. Unlike some ETL tools, Piperoni relies on in-memory transformation, and thus is ideal for manipulating complex, diverse non-"big"-data.Piperoni allows you to make and track atomic data transformations, ensures expected types are being passed from transformation to transformation, and allows you to easily see the state of the data at any point in time. Piperoni is a great tool for collaborative data pipelines, where visibility into data transformations is key.Getting Startedpiperoniis a framework for ETL and data pipeline work. To get started, first installpiperoni:pipinstallgit+ssh://git@github.com/CitrineInformatics/piperoni.gitDocumentationDetailed instructions on installation and usage can be found in the completepiperoni docsContributingThe following best practices are required for contributing.In this repo, we follow PEP8 standards (using Black) and include Docstrings in all of work.All functions should have unit testing.Best PracticesNever use branching code in aPipeline(e.g. if, else) without an explicit warning or failure. Particularly, do not use branching if the branches give rise to same or similar data.Do not usedeepcopy()in any operators; this will cause unexpected behavior.Keep transforms atomic! This is the reason for Piperoni. Don't be lazy.Stuck? Piperoni logs every transformation! Just set it to debug mode!Have intermediate states be optionally output by usingCheckpointsDo not use nestled Types when defining Types in your Operators (e.g.DictnotDict[str, str])Avoid hidden-states / adopt functional programming practices whenever possibleAvoid multiple versions of files for optioning. Adopt argparse or similar instead whenever possible.Use named variables and either avoid or fill in optional variables in function calls.Do not hard code column names or similar, even when the function only ever applies to a single column or instance.Have a trusted reference. Always compare to trusted reference after changes to the pipeline. Update the reference as needed.Flagging Bugs and Requesting New FeaturesWe funnel Bugs and Feature requests through Github issues. Create a new issue and select Bug Report or Feature Request (If you have neither a bug or feature request, open a regular issue). Add a concise title, fill in the template, and submit the issue.CitationsExample Band Gap data used in the example are from: Strehlow, W. H., & Cook, E. L. (1973). Compilation of energy band gaps in elemental and binary compound semiconductors and insulators. Journal of Physical and Chemical Reference Data, 2(1), 163-200.
Package: pipertool
pipertoolPiper is an open-source platform for data science and machine learning prototyping.FeaturesFully typed with annotations and checked with mypy,PEP561 compatibleAdd yours!InstallationpipinstallpipertoolExampleShowcase how your project can be used:frompipertool.exampleimportsome_functionprint(some_function(3,4))# => 7LicenseApacheCreditsThis project was generated withwemake-python-package. Current template version is:e2cd84edbe59fa23e7c97cd448b324f3744b10ac. See what isupdatedsince then.
Package: piper-tools
PiperA software manager for easydevelopmentanddistributionofPython code.The main features that Piper adds to Python are:Support for large-scale,multi-package projectsReproducibility(clear, transparentdependency management)Robustdevelopment-lifecycle, from blueprinting to distributionPiper is inspired by whatMavenis for Java and usesPipandVirtual Environments.Why PiperPython is greatin many things, particularly forscripting. But it is powerful enough to createcomplex softwaretoo. Still, when doing so, it lacks some robustness and quickness.Instead:Piper lets youforget about repetitive setup of Python projects, with creation of Virtual Environments, issues with imports, PYTHONPATH, folder structures. It does all of this for you.Piper ensures that when you run a Python script, all itsrequirements are implicitly installed. Along with reproducibility, this makes room foreasy collaborationbetween developers.Also, while with standard tools it's hard to have multiple packages, one requiring the other, with Piper is a matter offew YAML files. You cansplit your projectin several packages instead of having tons of requirements, code replication and/or single package-monoliths.All in all, Piper lets you designproper structure and modularityfor your code. No worries about imports and low-level stuff.Example of Piper ProjectYou can find an example of Piper projecthere.
Package: pipeserial
PipeSerialPipeSerial is a command-line utility to send some input from stdin to a serial device and return the output on stdout.A quick example:[tykling@container1 ~]$ echo -ne "AT\nATI" | sudo pipeserial -c 2 /dev/ttyU0.3
AT
OK
ATI
Quectel
EC25
Revision: EC25EFAR06A06M4G

OK

[tykling@container1 ~]$Same thing with debug enabled:[tykling@container1 ~]$ echo -ne "AT\nATI" | sudo pipeserial -c 2 -d /dev/ttyU0.3
2020-10-31 22:49:42 +0000 pipeserial DEBUG pipeserial.main():311:  Initialising the PipeSerial class
2020-10-31 22:49:42 +0000 pipeserial DEBUG pipeserial.__init__():54:  Configuring serial port {serialport} ...
2020-10-31 22:49:42 +0000 pipeserial DEBUG pipeserial.main():329:  Payload is 6 bytes
2020-10-31 22:49:42 +0000 pipeserial DEBUG pipeserial.main():335:  Output to expect: ['\r\nOK\r\n', '\r\nERROR\r\n']
2020-10-31 22:49:42 +0000 pipeserial DEBUG pipeserial.open():75:  Opening serial port...
2020-10-31 22:49:42 +0000 pipeserial DEBUG pipeserial.open():82:  Serial port opened OK!
2020-10-31 22:49:42 +0000 pipeserial DEBUG pipeserial.run():107:  Sending payload line: AT
2020-10-31 22:49:43 +0000 pipeserial DEBUG pipeserial.run():107:  Sending payload line: ATI
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.run():114:  Collecting output, looking for one of these regular expressions: ['\r\nOK\r\n', '\r\nERROR\r\n']
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.run():116:  Will stop collecting after 2 matches
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.run():120:  Found match: 'OK' (match number 1 of 2)
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.run():120:  Found match: 'OK' (match number 2 of 2)
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.run():127:  Done! Returning 64 bytes of output from serial device
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.close():133:  Closing serial port...
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.close():135:  Serial port closed
2020-10-31 22:49:44 +0000 pipeserial DEBUG pipeserial.main():355:  Got 9 lines of output from serial device /dev/ttyU0.3:
AT
OK
ATI
Quectel
EC25
Revision: EC25EFAR06A06M4G

OK

[tykling@container1 ~]$Read more on ReadTheDocs athttps://pipeserial.readthedocs.io/en/latest/
Package: pipestat
Pipestat standardizes reporting of pipeline results. It provides 1) a standard specification for how pipeline outputs should be stored; and 2) an implementation to easily write results to that format from within Python or from the command line. A pipeline author defines all the outputs produced by a pipeline by writing a JSON-schema. The pipeline then uses pipestat to report pipeline outputs as the pipeline runs, either via the Python API or command line interface. The user configures results to be stored either in aYAML-formatted fileor aPostgreSQL database.SeePipestat documentationfor complete details.Developer testsOptional DependenciesNote: to run the pytest suite locally, you will need to install the related requirements:cdpipestat

pipinstall-rrequirements/requirements-test.txtDatabase Backend Configuration for TestsMany of the tests require a postgres database to be set up otherwise many of the tests will skip.We recommend using docker:dockerrun--rm-it--namepipestat_test_db\-ePOSTGRES_USER=postgres\-ePOSTGRES_PASSWORD=pipestat-password\-ePOSTGRES_DB=pipestat-test\-p5432:5432\postgres
Package: pipestat-reader
pipestat_reader - web API generator for pipeline resultsPipestat reader is a Python package that is an extension ofpipestat.Pipestat can be used to collect pipeline results and store them in either a YAML-formatted file or in a database. If the results are stored in the database, then pipestat reader can be used to automatically generate a web API to query these results.UsageCreate apipestat.PipestatManagerobjectimportpipestatpsm=pipestat.PipestatManager(config="/path/to/config.yaml")Pass the objects to thepipestat_reader.PipestatReaderconstructorimportpipestat_readerpsr=pipestat_reader.PipestatReader(pipestat_managers=[psm])Generate aGraphQLschemagraphql_schema=psr.generate_graphql_schema()Use the schema in your web application, for example inFastAPI:fromfastapiimportFastAPIfromstarlette.graphqlimportGraphQLAppapp=FastAPI(title="test app")app.mount("/graphql",GraphQLApp(schema=graphql_schema,graphiql=True))Retrieve the results reported with pipestatByPOSTqueriescurl -X POST \-H "Content-Type: application/json" \-d '{"query": "{table_name{edges{node{column_name}}}}"}' \http://0.0.0.0:8000/graphql/Using the interactive GraphiQL interface athttp://0.0.0.0:8000/graphql/
Package: pipestream
PipestreamUsagePipestream provides a convenient way to "pipe" a given input through a series of classes or callable, giving each class the opportunity to inspect or modify the input and invoke the next callable in the pipeline:frompipestreamimportPipelinedefsum_one(value,next):returnnext(value+1)defmultiply_by_two(value,next):returnnext(value*2)Pipeline.send(1).through(sum_one,multiply_by_two).then(print)# output: 4As you can see, each invokable class or closure in the pipeline is provided the input and anextclosure. Invoking thenextclosure will invoke the next callable in the pipeline. As you may have noticed, this is very similar to middleware andchain of responsability pattern.When the last callable in the pipeline invokes thenextclosure, the callable provided to the then method will be invoked. Typically, this callable will simply return the given input.Of course, as discussed previously, you are not limited to providing functions to your pipeline. You may also provide classes with common method's between them. If a class is provided, the class method will be accessed by python std methodgetattr.frompipestreamimportPipelineuser=Pipeline.send(user).through(GenerateProfilePhoto,ActivateSubscription,SendWelcomeEmail).via('do').then(lambdavalue:value)Install$pipinstallpipestreamRunning tests$pytesttests/This package is inspired byLaravel's Pipeline Helper
Package: pipe-strip
Pipe StripInstall withpipx install pipe-strip(for an isolated environment) orpip install pipe-stripRun withpipe-stripThe ANSI art automatically scales to fit your terminal size.Cyclic mode (seepipe-strip --help) allows for infinite viewing of the Pipe Strip.LicenseTo the extent possible under law,Isaiah Odhnerhas waived all copyright and related or neighboring rights toPipe Strip.
This work is published from:United States.
Package: pipe-subprocess
pipe-subprocess' Connects input/ouput streams between multiple subprocesses easily just like shell pipe.
' Provides almost the same interface as python subprocess module with some exceptions.Installpip3installpipe-subprocessQuick how toJust pass commands to run function.>>>importpipesubprocessaspipesub>>>result=pipesub.run(["ls /usr/bin/","grep py","head -2"],stdout=pipesub.PIPE)>>>print(result.stdout)pbcopypydoc# Commands can be written just as *args.```python>>>result=pipesub.run("ls /usr/bin/","grep py","head -2")>>>print(result.stdout)pbcopypydocrun() quick glance.Userun()function to run multiple piped subprocesses. In more complecated case, use pipesubporcess.Popen.defrun(*cmdlist,shlex=True,stdin=None,input=None,stdout=None,stderr=None,capture_output=True,shell=False,cwd=None,timeout=None,check=False,encoding=None,errors=None,text=True,env=None,universal_newlines=None)It runs multiple process specified incmdlistand connects stdout and stdin in the order ofcmdlist.It returnsCompletedProcess()inputis written to stdin of the first command.Whentimeout([second] in float) is specified, it raisespipesubprocess.TimeoutExpired()when commands takes longer than that timeout.Ifcheck=True, it raisespipesubprocess.CalledProcessError()when there is a command with exit code not zero.
' See [README_details.md] for other options.class pipesubprocess.PipeSubprocessErrorAll exceptions of pipesubprocess module inherits this exception. So all exceptions of thie module can be catched by this exception.class pipesubprocess.PopenArgsIt represents the arguments of one process passed to *subproces.Popen().namename of the argument. The first args by default.See [README_details.md] for other attributes.class pipesubprocess.CompletedProcessThe return value fromrun(), representing the finish status of processes.popen_args_listList of PopenArgs instance.returncodesList of return codes. None if all commands have not finished yet.stdoutstdout of the last command.
,stderrgathred stderr of all the commands.check_returncodes()raisespipesubprocess.CalledProcessError()whenreturncodeshave any non-zero status.class pipesubprocess.TimeoutExpiredpopen_argsList of PopenArgs instance.timeouttimeout value specified.stdoutstdout of the last command.stderrgathred stderr of all the commands.
Package: pipes-vis
pipes_visA small, interactive GUI/visualizer tool for SPS spectra, powered by Adam Carnall's Bayesian Analysis of Galaxies for Physical Inference and Parameter EStimation(BAGPIPES)A"Mini-paper"about pipes_vis is published in the Research Notes of the AAS. Please cite this paper if you use pipes_vis in your publications.To install, typepip install pipes_vis. This package depends on Bagpipes, which has non-python packages like multinest as dependencies. If you only wish to use pipes_vis, you need not go through the process to get all non-python dependencies of bagpipes as they are only used for SED fitting rather than modelling. Just ignore complains from Bagpipes when importing. However, if you wish to use Bagpipes for SED fitting, follow the tutorial in Bagpipes' docs to get those non-python packages.You can find examples and documentations of the tool inthis jupyter notebookDependencies: bagpipes, numpy, matplotlib, astropy, scipy (visualizer still works even if you do not install multinest and pymultinest as instructed by bagpipes' installation)For requests regarding making the visualizer avaliable as a web interactive tool much likesengi, it will likely require rewriting large portions of the code in javascript or some other web-purposed languages. Please contact me if you are interested in doing this/want this to be done.
Package: pipetex2tex
CLI to convert LaTeX source with piped\input{|...}commands into
pure LaTeX.Installpip install pipetex2texUsagepipetex2tex manuscript.tex > manuscript_with_executed_commands.tex
cat manuscript.tex | pipetex2tex > manuscript_cmds.texOr in python:importpipetex2texasp2ttex="There's \inp{|python -c 'print(int(24*60*60*365.25))'} seconds in a year."print("source:",tex)print("out   :",p2t.convert(tex),'\n')tex="There's $\input { | ls -al ~ | wc -l }$ files/directories in your user directory."print("source:",tex)print("out   :",p2t.convert(tex))Results insource: There's \inp{|python -c 'print(int(24*60*60*365.25))'} seconds in a year.
out   : There's 31557600 seconds in a year.

source: There's $\input { | ls -al ~ | wc -l }$ files/directories in your user directory.
out   : There's $      62$ files/directories in your user directory.Dependenciespipetex2texonly uses the Python standard library.LicenseThis project is licensed under theMIT
License.
Package: pipeto
code like this:pipe(1) | float | str | list | done!tested on 2.7, 3.2, 3.3InstallpipinstallpipetoAPIpipe(arg)generate a pipable object pipe to next function.@param : arg {mixed}done(arg)get the actural value out of pipable objectcompose(fn)Compose functions. Can be used as a decorator.@alias : composable@param : fn {callable}examplefrompipetoimport*importoperatorasopinc=lambdax:x+1double=lambdax:x+x# pipepipe(1)|float|str|list|done# == ['1', '.', '0']pipe(2)|inc|done# == 3pipe(2)|inc|double|done# == 6pipe([1,2,3])|sum|done# == 6# composenewfn=compose(inc)|doublenewfn(2)# == double(inc(2))
Package: pipetography
PipetographyNipype and mrtrix3 based pre-/post- processing pipeline for brain diffusion-MRI and generation of structural connectomes of the brain.The pre-processing pipeline has been updated to reflect what's seen in the optimalDESIGNER pipelineand onmrtrix3 cloud apps on brainlife.io.The post-processing pipeline, including co-registration of atlases and connectome generation is now included in theconnectomesmoduleInstallSince most usages will be on HPC resources, Ihighly recommendthat you use theSingularitydefinition file in the repository instead of installing the Python module.Singularity:This is a large image, you will need to set the following environment variables to somewhere other than/tmpif you want to avoid memory errors:export SINGULARITY_TMPDIR={YOUR DESTINATION DIR}export SINGULARITY_LOCALCACHEDIR={YOUR DESTINATION DIR}export SINGULARITY_CACHEDIR={YOUR DESTINATION DIR}export SINGULARITY_PULLFOLDER={YOUR DESTINATION DIR}Build the singularity image with thesingularity.deffile provided in Github, you will need to havesudopermissions to perform singularity build. If you run into memory problems, consider building as a sandbox at first with the-sflag.sudo singularity build {image_file_name}.sif singularity.defOR pull the built singularity image from cloud library:singularity pull --arch amd64 library://axiezai/pipetography/pipetography:latestDocker:Docker often encounters permission problems on HPC resources, using the docker container is only recommended if you havesudopermissions to the machine.Use theDockerfilein this repo to build your own Docker container.Usage:To run the singularity container interactively, you will need a few flags:-eflag for a clean environnment upon container start-Bflag to bind your BIDS directory into the container.Example:singularity shell -e -B <BIDS_DIR>:<SINGULARITY_BIDS_DIR> {Path to singularity .sif}will start an interactive shell.singualrity exec -e -B <BIDS_DIR>:<SINGULARITY_BIDS_DIR> {Path to singularity .sif} python dwi_pipeline.pywill execute the input python code. Seepipelinefor an example python script that preprocesses BIDS DWI data.Creating your own environment and installpipetographyas a Python module:pip install pipetographySincepipetographyis aNipypewrapper aroundmrtrix3,ANTs, andFSL, you have to follow their installation instructions and set them up appropriately on your machine as well:mrtrix3 v3.0.0ANTsFSLEverything listed in this section is included in the Singularity and Docker containers.The pipeline:Currently supports acquisitions with no reverse phase encoding (-rpe_none)  and reverse phase encoding in all DWI directions (-rpe_all) options. Seepipelinefor preprocessing details, andconnectomesfor postprocessing details.
Package: pipettewrapper
PipetteWrapperPipetteWrapper is used to wrap opentrons multichannel Pipette objects to allow multichannels to act as single channels.CURRENTLY REQUIRES A MODIFICATION TO LINE 67 INopentrons/api/calibration.py:CHANGE:inst.pick_up_tip(container._container[0])TO:inst.pick_up_tip(location=container._container[0])Examplefrom opentrons import labware, instruments
tip_rack = labware.load('opentrons-tiprack-300ul', slot='4')
pipette = PipetteWrapper(instruments.P50_multi(mount='left', tip_racks=[tip_rack]))
plate = labware.load('96-flat')
pipette.distribute(50, plate.wells('A1'), plate.wells('A2'), num_tips=4)  # Uses 4 tips
pipette.transfer(50, plate.wells('A1'), plate.wells('A2'))  # Defaults behaviour (8 tips)
Package: pipettor
Pipettor Overviewpipettor - robust, easy to use Python package for running Unix process pipelinesFeaturesCreating process pipelines in Python is either complex (e.g.subprocess),
or not robust (e.g.os.system()).  This package aims to address these shortcomings.Command pipelines are simply specified as a sequence of commands, with each
command represented as a sequence of arguments.Failure of any process in the pipeline results in an exception, withstderrincluded in the exception.Pipelinestdin/stdout/stderrcan be passed through from parent process,
redirected to a file, or read/written by the parent process.Asynchronous reading and writing to and from the pipeline maybe done without risk of
deadlock.Pipeline can run asynchronously or block until completion.Popen-style File-like objects for reading or writing a pipeline.Documentation:https://pipettor.readthedocs.org.History1.0.0 (2023-06-29)Don't use a process group; as it caused signals to not get propagated.  Processes are explicitly waited for by pid, so this will not consume the exit of other process not create by this module.0.8.0 (2023-02-05)make most optional arguments require keyword form to help prevent errors, especially if open() options are assumedadded more functions to make Popen objects file-like objects0.7.0 (2023-01-06)don't fail if invalid UTF-8 characters are written to capture stderr0.6.0 (2022-11-16)remove use of deprecated pipes module0.5.0 (2020-12-25)Removed Python-2 support.Switch to using subprocess as a base rather interface directly
with Unix system calls.  This lets subprocess deal with
various issues dealing with the Python interpreter environment.0.4.0 (2018-04-21)Allow passing through universial newline mode for PY2.Fix bug with not using specified log level.0.3.0 (2018-02-25)added open-stying buffering, encoding, and errors optionssource cleanup0.2.0 (2017-09-19)Simplified and log of info and errors levels by removing logLevel options.Improvements to documentation.0.1.3 (2017-06-13)Documentation fixes0.1.2 (2017-06-11)First public release on PyPI.
Package: pipeup
PipeUp is a command line tool for piping to a webpage.$tail-f/var/log/nginx/access.log|pipeupInstallationTo install PipeUp, simply:$pipinstallpipeupUsagePipe the output of a command topipeup. See available options by runningpipeup--help$command|pipeupServerTo run your own server, install PipeUp from source:$gitclonegit@github.com:nathancahill/pipeup.git$cdpipeup$pipinstall-rrequirements-server.txt$cdpipeup/server$cpconfig.example.pyconfig.pyEditconfig.pywith the complete URL of your server. Then run the Tornado server:$pythonserver.pyTo use thepipeupclient with your server, pass the websocket URL as--server:$command|pipeup--serverws://127.0.0.1:8888/wsFor a more permanent installation, put Tornado behind Nginx using thenginx.conffile.ContributingPull requests are gladly accepted. Keep it simple.
Package: pipe-utils
Pipe UtilsPython with pipes, utils, and pipe utils.Installpip install pipe-utilsDocshttps://pipe-utils.rtfd.ioExamplefrompipe_utilsimport*words="I just think pipes are neat!"result=(Pipe(words)|obj.lower()|obj.replace("!","")|obj.split()|group_by(len)|sorted_dict()).get()print(result)#  {1: ['i'], 3: ['are'], 4: ['just', 'neat'], 5: ['think', 'pipes']}And, if you're feeling dangerous, override builtin functions
likefilter,map, andallby importing frompipe_utils.override:frompipe_utils.overrideimport*data=[[1,-3,4],[1,2,3],[2,3,4],[5,-1,4]]result=(Pipe(data)|filter(all(it>=0))|map(sum_by(it*it))|as_list).get()print(result)# [14, 29]And, if you're feeling extra dangerous, you can use arrow syntax to clean up
some of those brackets (and thePpipe alias):frompipe_utils.overrideimport*data=[[1,-3,4],[1,2,3],[2,3,4],[5,-1,4]]result=(P>>data|filter>>all(it>=0)|map>>sum_by(it*it)|as_list).get()print(result)# [14, 29]