Saravanan666/pypi-org huggingface dataset:
Package: prefect-soda-core
prefect-soda-coreWelcome!Prefect 2.0 collection for Soda CoreGetting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.Installationprefect-soda-coreis based onsoda-core.Assoda-corerequires you to specify the right option for your database, so doesprefect-soda-core.I.e. to useprefect-soda-corewith Snowflake, run the following:pipinstallprefect-soda-core[snowflake]You can find the list of supported options insetup.py.Please note that since this integration is built on top of Soda CLI, it is not possible to run data quality checks using Spark.Write and run a flowfromprefectimportflowfromprefect.contextimportget_run_contextfromprefect_soda_core.soda_configurationimportSodaConfigurationfromprefect_soda_core.sodacl_checkimportSodaCLCheckfromprefect_soda_core.tasksimportsoda_scan_execute@flowdefrun_soda_scan():soda_configuration_block=SodaConfiguration(configuration_yaml_path="/path/to/config.yaml")soda_check_block=SodaCLCheck(sodacl_yaml_path="/path/to/checks.yaml")# Using the flow_run_name as the name of the file to store the scan resultsflow_run_name=get_run_context().flow_run.namescan_results_file_path=f"{flow_run_name}.json"returnsoda_scan_execute(data_source_name="my_datasource",configuration=soda_configuration_block,checks=soda_check_block,variables={"var":"value"},scan_results_file=scan_results_file_path,verbose=True,return_scan_result_file_content=False,shell_env={"SNOWFLAKE_PASSWORD":"********"})run_soda_scan()ResourcesIf you encounter any bugs while usingprefect-soda-core, feel free to open an issue in theprefect-soda-corerepository.If you have any questions or issues while usingprefect-soda-core, you can find help in either thePrefect Discourse forumor thePrefect Slack community.DevelopmentIf you'd like to install a version ofprefect-soda-corefor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/sodadata/prefect-soda-core.gitcdprefect-soda-core/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-spark-on-k8s-operator
prefect-spark-on-k8s-operatorVisit the full docshereto see additional examples and the API reference.Prefect integrations for orchestrating and monitoring apache spark jobs on kubernetes using spark-on-k8s-operator.Welcome!prefect-spark-on-k8s-operatoris a collection of Prefect flows enabling orchestration, observation and management ofSparkApplicationcustom kubernetes resources defined according to spark-on-k8s-operator CRD v1Beta2 API Spec.Jump toexamples.ResourcesFor more tips on how to use tasks and flows in a Collection, check outUsing Collections!InstallationYou need to configure the kubernetes credentials as perprefect-kubernetesdocumentation.Installprefect-spark-on-k8s-operatorwithpip:pipinstallprefect-spark-on-k8s-operatorRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These flows are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.Example UsageSpecify and run a SparkApplication from a yaml fileimportasynciofromprefect_kubernetes.credentialsimportKubernetesCredentialsfromprefect_spark_on_k8s_operatorimport(SparkApplication,run_spark_application,# this is a flow)app=SparkApplication.from_yaml_file(credentials=KubernetesCredentials.load("k8s-creds"),manifest_path="path/to/spark_application.yaml",)if__name__=="__main__":# run the flowasyncio.run(run_spark_application(app))FeedbackIf you encounter any bugs while usingprefect-spark-on-k8s-operator, feel free to open an issue in theprefect-spark-on-k8s-operatorrepository.If you have any questions or issues while usingprefect-spark-on-k8s-operator, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to star or watchprefect-spark-on-k8s-operatorfor updates too!ContributingIf you'd like to help contribute to fix an issue or add a feature toprefect-spark-on-k8s-operator, pleasepropose changes through a pull request from a fork of the repository.Here are the steps:Fork the repositoryClone the forked repositoryInstall the repository and its dependencies:pip install -e ".[dev]"Make desired changesAdd testsInsert an entry toCHANGELOG.mdInstallpre-committo perform quality checks prior to commit:pre-commit installgit commit,git push, and create a pull request
Package: prefect-stitch
prefect-stitchWelcome!Prefect collection to interact with StitchGetting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-stitchwithpip:pipinstallprefect-stitchWrite and run a flowfrompydanticimportSecretStrfromprefectimportflowfromprefect_stitch.credentialsimportStitchCredentialsfromprefect_stitch.tasksimport(start_replication_job)@flowdefexample_flow():creds=StitchCredentials(access_token=SecretStr("my_secret_token"))start_replication_job(credentials=creds,source_id=1234)example_flow()ResourcesIf you encounter any bugs while usingprefect-stitch, feel free to open an issue in theprefect-stitchrepository.If you have any questions or issues while usingprefect-stitch, you can find help in either thePrefect Discourse forumor thePrefect Slack community.DevelopmentIf you'd like to install a version ofprefect-stitchfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/AlessandroLollo/prefect-stitch.gitcdprefect-stitch/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-streamline
prefect streamlineprefect_streamline is an opinionated extension that provides helpers to deploy and test Prefect flows quickly and easily.With its deployment and testing features, you can easily ensure the quality and reliability of your workflows before deploying them.Getting startedUsageDeploy flows using decoratorAccelerate flow testing using native python invocationLatest versionDevelopper guidelineInstall development environmentupdate the locked dependenciesRun the continuous integration processContributorsLicenseGetting startedpipinstallprefect_streamlineUsageDeploy flows using decoratorprefect_streamlineallows you to configure the deployment of flows with a decorator.
Vous pouvez définir plusieurs déploiements pour un même flow.myapp/flow.pyfromprefectimportflowfromprefect_streamlineimportdeploybook@deploybook.register(interval=90,name="hello2")@deploybook.register(cron="*/3 * * * *")@deploybook.register(name="manual")@flow(name="main.myflow")defmyflow()->int:return43the prefect-streamline cli browses all the subfolders of your application.# deploy all the flows register into myapp/flow.pyprefect-streamlinedeploymyapp/flow.py# search all flow records in all myapp modules.prefect-streamlinedeploy--discoversrc/myappAccelerate flow testing using native python invocationfromprefect_streamlineimportflowtestdeftest_test_flow_should_handle_the_logger():withflowtest.use_native_runner():assertflowtest.fn(myflow)()==12@flow()defmyflow()->int:logger=get_run_logger()return12Latest versionYou can find the latest version to ...gitclonehttps://github.com/FabienArcellier/prefect_streamline.gitDevelopper guidelinepoetry shellInstall development environmentUse make to instanciate a python virtual environment in ./venv and install the
python dependencies.poetryinstallupdate the locked dependenciespoetryupdateRun the continuous integration processBefore commit or send a pull request, you have to execute the continuous integration process.poetryrunalfredciContributorsFabien ArcellierLicenseMIT LicenseCopyright (c) 2018-2022 Fabien ArcellierPermission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Package: prefect-supporter
prefect_supporterprefect_supporter provide services to supporter to manage PrefectUsageDeleteflow runhistory in Database. It also deletestask runcreate your deploymentexamplefromdatetimeimportdatetime,timedeltafromprefect.settingsimportPREFECT_API_URLfromprefect.deploymentsimportDeploymentfromprefect.server.schemas.schedulesimportCronSchedulefromprefect.server.schemas.statesimportStateTypefromprefect_supporter.flowimportclear_db_historyfromprefect_supporter.flow.clear_db_historyimportclear_db_history_flowfromprefect_supporter.model.clearing_dbimportFlowRunClearingDeployment.build_from_flow(flow=clear_db_history_flow,name="clear_db_history_deployment",schedule=CronSchedule(cron="0 * * * *",),work_queue_name="default",tags=["prefect-management"],entrypoint=f"{clear_db_history.__file__}:clear_db_history_flow",parameters={"frc":FlowRunClearing(api_url=PREFECT_API_URL.value(),state_list=[StateType.COMPLETED],before_dt=(datetime.now()-timedelta(days=7)),after_dt=datetime(year=1970,month=1,day=1,hour=0,minute=0))},apply=True)ParameterFlowRunClearingmodelapi_url : your prefect api address, ex)http://localhost:4200/apistate_list: state list to delete ex) "COMPLETED", "FAILED"before_dt : time range to delete data to be startedafter_dt : time range to delete data  to be ended
Package: prefect-transform
prefect-transformWelcome!Collection of tasks to interact with Transform metrics catalogGetting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-transformwithpip:pipinstallprefect-transformWrite and run a flowfromprefectimportflowfromprefect_transform.tasksimport(create_materialization)@flowdeftrigger_materialization_creation():create_materialization(api_key="<your Transform API key>",mql_server_url="<your MQL Serverl URL>",materialization_name="<name of the materialization>",wait_for_creation=False)trigger_materialization_creation()ResourcesIf you encounter any bugs while usingprefect-transform, feel free to open an issue in theprefect-transformrepository.If you have any questions or issues while usingprefect-transform, you can find help in either thePrefect Discourse forumor thePrefect Slack community.DevelopmentIf you'd like to install a version ofprefect-transformfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/AlessandroLollo/prefect-transform.gitcdprefect-transform/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-twitter
prefect-twitterVisit the full docshereto see additional examples and the API reference.Prefect integrations for interacting with Twitter.ResourcesFor more tips on how to use tasks and flows in a Collection, check outUsing Collections!InstallationInstallprefect-twitterwithpip:pipinstallprefect-twitterRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2. For more information about how to use Prefect, please refer to thePrefect documentation.Write and run a flowTweets an update with just text.fromprefectimportflowfromprefect_twitterimportTwitterCredentialsfromprefect_twitter.tweetsimportupdate_status@flowdefexample_update_status_flow():twitter_credentials=TwitterCredentials(consumer_key="consumer_key",consumer_secret="consumer_secret",access_token="access_token",access_token_secret="access_token_secret")status_id=update_status(twitter_credentials,status="A Prefect Tweet!")returnstatus_idexample_update_status_flow()FeedbackIf you encounter and bugs while usingprefect-twitter, feel free to open an issue in theprefect-twitterrepository.If you have any questions or issues while usingprefect-twitter, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to star or watchprefect-twitterfor updates too!DevelopmentContributingIf you'd like to help contribute to fix an issue or add a feature toprefect-twitter, pleasepropose changes through a pull request from a fork of the repository.Here are the steps:Fork the repositoryClone the forked repositoryInstall the repository and its dependencies:pip install -e ".[dev]"Make desired changesAdd testsInsert an entry toCHANGELOG.mdInstallpre-committo perform quality checks prior to commit:pre-commit installgit commit,git push, and create a pull request
Package: prefect-vault
prefect-vaultWelcome!Prefect blocks for HashiCorp Vault.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-vaultwithpip:pipinstallprefect-vaultCreate VaultSecret block with token authenticationfromprefect_vaultimportVaultSecret,VaultTokenvault_secret=VaultSecret(vault_auth=VaultToken(vault_url='http://myvault:8200',token='my_secret_token',),)awaitvault_secret.save('my-vault-secret')Create VaultSecret block with approle authenticationfromprefect_vaultimportVaultAppRole,VaultSecretvault_secret=VaultSecret(vault_auth=VaultAppRole(vault_url='http://myvault:8200',role_id='my_role_id',secret_id='my_secret_id',),)awaitvault_secret.save('my-vault-secret')Use VaultSecret in Prefect flowfromprefectimportflowfromprefect_vaultimportVaultSecret@flowasyncdefuse_vault_secret():vault_secret=awaitVaultSecret.load('my-vault-secret')secret_value=vault_secret.get_secret('path/to/my/secret')['secret_key']ResourcesIf you encounter any bugs while usingprefect-vault, feel free to open an issue in theprefect-vaultrepository.If you have any questions or issues while usingprefect-vault, you can find help in either thePrefect Discourse forumor thePrefect Slack community.DevelopmentIf you'd like to install a version ofprefect-vaultfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/pbchekin/prefect-vault.gitcdprefect-vault/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-yaml
Prefect YAMLPackage to run prefect with YAML configuration. For further details, please refer
to thedocumentationInstallationInstall this via pip (or your favourite package manager):pip install prefect-yamlUsageRun the command lineprefect-yamlwith the specified configuration
file.For example, the following YAML configuration is located inexamples/simple_config.yaml.metadata:
  output:
    directory: .output

task:
  task_a:
    caller: math:fabs
    parameters:
      - -9.0
    output:
      format: json
  task_b:
    caller: math:sqrt
    parameters:
      - !data task_a
    output:
      directory: null
  task_c:
    caller: math:fsum
    parameters:
      - [!data task_b, 1]Run the following command to generate all the task outputs to the
directory.outputin the running directory.prefect-yaml-cexamples/simple_config.yamlThe output directory contains all the task outputs in the specified
format.%tree.output
.output
├──task_a.json
└──task_c.pickle0directories,2filesThe expected behavior is toruntask_ato dump the valuefabs(-9.0)to the output directory in JSON format,runtask_bto get the valuesqrt(9.0)(from the output oftask_a)runtask_cto dump the valuefsum([3.0, 1.0])to the output directory in pickle format.As the output directory intask_bis overridden asnull, the output oftask_bis passed totask_cin memory. Also, the output format intask_cis not specified so it is dumped in default format (pickle).For further details, please see the sectionconfigurationin the documentation.ConfigurationThe output section defines how the task writes and loads the task return. The section inmetadataapplies for all tasks globally while that in eachtaskoverrides the global parameters.For further details, please see thedocumentationfor parameter definitions
in each section.OutputThe default output format is either pickle (default) or JSON, while users
can define their own output format.For example, if you would like to usepandasto load and dump the parquet file
in pyarrow engine by default, you can define the configuration like below.metadata:
  format: parquet
  dump-caller: object.to_parquet
  dump-parameters:
    engine: pyarrow
  load-caller: pandas:read_parquet
  load-parameters:
    engine: pyarrowAll the output parameters, like directory, dumper and loaders, can be overridden
in the task level. You can also specify which tasks to export to the output
directory, while the others to only be passed down to downstream in memory.For further details, please see theoutputsection in documentation.RoadmapCurrently the project is still under development. The basic features are
mostly available while the following features are coming soonMulti cloud storage supportSubtasks supported in each taskContributingAll levels of contributions are welcomed. Please refer to thecontributingsection for development and release guidelines.
Package: prefer
What is it? [![CircleCI](https://img.shields.io/circleci/project/github/LimpidTech/prefer.py.svg?style=flat-squared)](https://circleci.com/gh/LimpidTech/prefer.py) [![Coveralls](https://img.shields.io/coveralls/LimpidTech/prefer.py.svg?style=flat-sqared)](https://coveralls.io/github/LimpidTech/prefer.py)-----------Prefer is a library for helping you manage application configurations whileproviding your users the flexibility of using whatever configuration formatfits their needs.It provides a set of interfaces which provide standard methods forreading arbitrary project configuration data. This can vary from simple caseslike JSON, to more complicated examples - such as retreiving configuration datafrom a database.How do I use it?----------------Firstly, you'll want to install the module. This can be done easily with`easy_install` or `pip`.```sheasy_install prefer```Prefer is fairly simple to use. A basic use case might be that you have thefollowing JSON configuration in *settings.json*:```json{"auth": {"username": "user","password": "pass"}}```You can load these settings with the following code using promises:```pythonimport preferconfiguration = await prefer.load('settings')username = configration.get('auth.username')```You will notice that prefer only required 'settings' as the filename. It isrecommended to be given without a path or extension, because prefer takes careof looking through the filesystem for configuration files. On both Unix andWindows systems, it will look in all of the standard folders, as well as someconventional places where people like to put their configurations.Ordering matters, so having a file in `./settings.json` as well as another in`/etc/settings.json` is still reliable. The configuration in `./settings.json`will be used first. Prefer doesn't care what format your user writes yoursettings in, so they can also use `settings.yaml` if they like.Supported configuration formats-------------------------------Along with being fully configurable to support any arbitrary data source you'dlike, the following types of data can immediately be used as configuration formatsupon installation of prefer:- JSON- YAMLWhy asyncronous?----------------In order to supply a more-simple method of getting the configuration, manyconfiguration tools prefer to provide a blocking method of retrieving a projectconfiguration. One goal of prefer is to ensure that we aren't limiting users tospecific use-cases - and some projects require real-time, dynamic updating oftheir configuration. Prefer provides all it's interfaces as asyncronousfunctions in order to provide that possibility without the requirement thatthose actions are blocking.[cov]: http://monokro.me/projects/prefer/coverage.html[bs]: https://travis-ci.org/LimpidTech/prefer.png?branch=master "Build Status"[j5]: http://json5.org/ "json5 - JSON for the ES5 era"
Package: preferred-pictures
PreferredPictures Python Client LibraryThePreferredPicturesPython library provides a convenient way to call thePreferredPicturesAPI for applications written in Python.View the documentation of this moduleView the documentation about the PreferredPicture's APILearn more about what PreferredPictures can do.Installation$ pip install preferred_picturesUsageThe package needs to be configured with your account's identity and
secret key, which is available in the PreferredPictures interface.frompreferred_picturesimportClientpp=Client("testidentity","secret123456")url=pp.create_choose_url(choices=["https://example.com/image-red.jpg","https://example.com/image-green.jpg","https://example.com/image-blue.jpg"],tournament="test-tournament")# Or more using a prefix and suffix for choicesurl=pp.create_choose_url(choices=["red","green","blue"],choices_prefix="https://example.com/image-",choices_suffix=".jpg",tournament="test-tournament")# The URL returned will appear to be something like:## https://api.preferred-pictures.com/choose-url?choices%5B%5D=red&choices%5B%5D=green&choices%5B%5D=blue&tournament=test-tournament&expiration=1597011644&uid=362511d6-997f-452b-afee-8e46331da04a&ttl=600&identity=testidentity&signature=7c6d305405b4518876543435ce5657ee820cad0601359fa173a854dc0dfd6ea1#LicenseThis client uses the MIT license.
Package: prefetchcarve
Python script to carve Windows Prefetch artifacts from arbitrary binary dataDescriptionThe Windows application prefetch mechanism is in place to offer performance benefits when launching applications. It’s also one of the more beneficial forensic artifacts regarding evidence of applicaiton execution. prefetch-carve.py provides functionality for carving prefetch artifacts from binary data - such as unallocated disk space, raw memory images, etc. prefetch-carve.py will output to the specified file, and supports multiple output formats.Supported Prefetch TypesWindows 10 Prefetch files are compressed, and are unable to be carved from disk in this manner. All other Prefetch formats are supported (Windows XP - Windows 8.1)Command-Line Optionsoptional arguments:
  -h, --help            show this help message and exit
  -f FILE, --file FILE  Carve Prefetch files from the given file
  -o OUTFILE, --outfile OUTFILE
                        Write results to the given file
  -c, --csv             Output results in csv format
  -m, --mactime         Output results in mactime format
  -t, --tln             Output results in tln format
  -s SYSTEM, --system SYSTEM
                        System name (use with -t)TestingThorough teseting is still underway. I plan to integrate this project with Travis CI shortly.InstallationUsing setup.py:python setup.py installUsing pip:pip install prefetchcarve