Saravanan666/pypi-org huggingface dataset:
Package: dafontdownloader
dafontdownloaderA python script to download fonts directly fromdafontsite.InstallFirst wayTo avoid usingsudo, installation requires these steps:Usepipto install the package with all dependencies required:pip install dafontdownloader --userUseexportto persistent use of dafontdownloader script:echo "export PATH=${PATH}:"$HOME"/.local/bin" >> "$HOME"/.bashrcUsesourceto start using script in command line:source .bashrcSecond WayDownload this repository, or usegit clone https://github.com/resilientcod/dafontdownloader.gitOpen repository directory.Usepython3 setup.py installFollow the steps 2 and 3, to persistent use of dafontdownloader script.Third Way (withsudo)Attention:use at your own risk!sudo pip install dafontdownloaderThis allow install inbindirectory, making it unnecessary to use steps 2 and 3 in the first way.UsageAfter installation, typedafontdownloaderin the terminal to verify that everything is correct,
as in the image:Then use:dafontdownloader <name-font>to download and install fonts from dafont site.Example:dafontdownloader Nemesis GrantAttention:Make sure you type the name correctly!More infoThe fonts are installed in the user's local.fontsdirectoryEnjoy!
Package: daga
dagadagaes un acrónimo deData Augmentation with Genetic Algorithm; este es un módulo de Python para el aumento de datos basado en la librería SDV. Este tiene como objetivo obtener la distribución correspondiente a cada variable y así conseguir mejores resultados en la generación de datos sintéticos. Para ello hace uso de un algortimo genético mono objetivo.El projecto se empezó en 2022 porAne MartínezyElene Astondoacomo trabajo para la universidad.Estructura de carpetassrc/daga/__ init __.pyfunctions.pydist/En esta carpeta se almacenan los archivos de distribución.LICENSEpyproject.tomlREADME.mdInstalaciónDependenciasdaga requiere:python (>=3.9)pandas (>=1.5.0)opencv-python (>=4.6.0)matplotlib (>=3.6.0)sdv (>=0.17.1)scikit-learn (>=1.1.2)Instalación para el usuarioUsandopip:Primero vamos a actualizar pip a la última versión:$ pip install --upgrade pipUna vez actualizado este comando procederemos a instalar la libreríadaga:$ pip install dagaLinks importantesRepositorio del código fuente:https://github.com/anemartinez1/pypi_libraryDescargar versiones:https://pypi.org/project/daga/Seguimiento de problemas:https://github.com/anemartinez1/pypi_library/issues
Package: dagapp
dagappMaking apps from DAGs by just snapping your fingersTo install:pip install dagappWhat's a DAG?What's an app?Enough theory. Here's how it works...A simple exampleFirst make a DAGfrommeshed.dagimportDAGdefb(a):return2**adefd(c):return10-(5**c)defresult(b,d):returnb*ddag=DAG((b,d,result))Then make an appfromdagapp.baseimportdag_appfromfunctoolsimportpartialdags=[dag]if__name__=="__main__":app=partial(dag_app,dags=dags)app()Then run the app>>> streamlit run example.py... and this is what you getA more complicated exampleLet's say we want to create two different DAGs and view them on the same streamlit page, keep all the non-root nodes static and represent the number inputs for one of the DAGs as sliders. We will use functions defined inconfigs_example.pyfor this example.Define the DAGsTo start we can create our DAGs just like in the previous example.profit_dag=DAG((user_clicks,rev,cost,profit))revenue_dag=DAG((partners,clicks,revenue))dags=[profit_dag,revnue_dag]Define the configsNext we can define some configs for these DAGs. These configs should be a list of dictionaries, with each dictionary representing the configs for each DAG. Each config must contain a dictionaryarg_typesthat matches each of the root nodes in the DAG to an input type (currently the options are: num, slider, text, list, dict). Ifarg_typesis not explicitly defined, then it will try to infer from type annotations in the function definitions, and then default to num. If slider is defined as thearg_typefor any of the root nodes, then another dictionaryrangesmust be defined that matches each of the nodes designated to be slider with a min and max value for that slider. The configs for this example can be seen below.configs=[dict(arg_types=dict(a="num",b="num",cost_per_click="num",revenue_per_click="num",),),dict(arg_types=dict(max_partners="slider",cost_per_click="slider",price_elasticity="slider",partners="slider",clicks_per_partner="slider",),ranges=dict(max_partners=[0,2000],cost_per_click=[0.0,1.0],price_elasticity=[0,200],partners=[0,1500],clicks_per_partner=[0.0,10.0],),),]Make the appWe can now make our app in a similar manner as the previous example, this time defining our configs, as well as definingStaticPageFuncas our page factory to keep all non-root nodes static.from dagapp.page_funcs import StaticPageFunc

if __name__ == "__main__":
    app = partial(dag_app, dags=dags, page_factory=StaticPageFunc, configs=configs)
    app()Run the app>>> streamlit run configs_example.py... and this is what you get
Package: dagather
Dagatherdagather (directedacyclicgather) is a new way to plan out and schedule asynchronous tasks. The tasks are organized, with each task specifying the tasks that come before it. The tasks are then run in topological order, ensuring that each operation will start as soon as it is able to without waiting for routines it does not need.fromasyncioimportsleepfromdagatherimportDagatherfoo=Dagather()@foo.register# add a new task to the task listasyncdefa():awaitsleep(1)return12@foo.registerasyncdefb(a):# we now specify that a is a requirement for this task,# meaning that b will not be called until a has finished.# during runtime, a's value will be the return value of the# a taskasserta==12awaitsleep(2)@foo.registerasyncdefc(a):awaitsleep(1)return'testing'@foo.registerasyncdefd():awaitsleep(1)@foo.registerasyncdefe(d,c):awaitsleep(1)result=awaitfoo()# when foo is called, it runs each of its registered tasks# as soon as all its dependencies are finished.# once all the tasks are finished, it will return a dict# mapping each task to its return value.assertresult=={a:12,b:None,c:'testing',d:None,e:None}
Package: dag-cbor
This is a fully compliant Python implementation of theDAG-CBOR codec, a subset of theConcise Binary Object Representation (CBOR)supporting theIPLD Data Modeland enforcing a unique (strict) encoded representation of items.ContentsInstallUsageEncoding and decodingRandom DAG-CBOR dataAPIContributingLicenseInstallYou can install the latest release fromPyPIas follows:$pipinstall--upgradedag-cborUsageWe suggest you import DAG-CBOR as follows:>>> import dag_cborBelow are some basic usage examples, to get you started: for detailed documentation, seehttps://dag-cbor.readthedocs.io/Encoding and decoding>>> dag_cbor.encode({'a': 12, 'b': 'hello!'})
b'\xa2aa\x0cabfhello!'
>>> dag_cbor.decode(b'\xa2aa\x0cabfhello!')
{'a': 12, 'b': 'hello!'}Random DAG-CBOR data>>> import pprint # pretty-printing
>>> custom_opts = dict(min_codepoint=0x41, max_codepoint=0x5a, include_cid=False)
>>> with dag_cbor.random.options(**custom_opts):
...     for d in dag_cbor.random.rand_dict(3):
...             pprint.pp(d)
...
{'BIQPMZ': b'\x85\x1f\x07/\xcc\x00\xfc\xaa',
 'EJEYDTZI': {},
 'PLSG': {'G': 'JFG',
          'HZE': -61.278,
          'JWDRKRGZ': b'-',
          'OCCKQPDJ': True,
          'SJOCTZMK': False},
 'PRDLN': 39.129,
 'TUGRP': None,
 'WZTEJDXC': -69.933}
{'GHAXI': 39.12,
 'PVUWZLC': 4.523,
 'TDPSU': 'TVCADUGT',
 'ZHGVSNSI': [-57, 9, -78.312]}
{'': 11, 'B': True, 'FWD': {}, 'GXZBVAR': 'BTDWMGI', 'TDICHC': 87}APIFor the full API documentation, seehttps://dag-cbor.readthedocs.io/ContributingPlease seeCONTRIBUTING.md.LicenseMIT © Hashberg Ltd.
Package: dag-dq-generator
# dag-dq-generator
DPaaS Airflow DAG (Dynamic Acyclic Graph) and DQ (Data Quality) generator.dag-dq-generatoris a DPaaS [Apache Airflow](https://github.com/apache/incubator-airflow) Airflow DAG (Dynamic Acyclic Graph) and DQ (Data Quality) generator  from YAML configuration files.
- [Usage](#usage)
- [YAML Definition](#yaml-definition)
- [Benefits](#benefits)
- [Contributing](#contributing)## Usage
### Setupdag-dq-generatorrequires Python 3.6.0+. To set up your environment, you can runsh build.shwhich installs the required Python packages and run the generator program. Otherwise, you can runpip install -r requirements.txtto install the required Python packages and runpython dag_generator.pywith the following parameters:
*–config-pathdefines the path to the configurations folder. Defaults to./configs/*–dag-storage-pathdefines the path to the folder where generated DAGs will be stored. Defaults to./dags/*–dq-storage-pathdefines the path to the folder where DQ SQL files will be stored. Defaults to./sql/## YAML Definition## BenefitsConstruct DAGs without knowing PythonConstruct DAGs without learning Airflow primitivesAvoid duplicative code## ContributingContributions are welcome! Just submit a Pull Request or Github Issue. Feel free to join the discussions on the#dag-dq-generatorSlack channel.
Package: dagdshort
dagdshortdagdshortis a Python based multithreaded URL shortener with a memory-cache. It uses theda.gdservice.As a disclaimer, this is an unofficial package and it has no association with da.gd. If you use this package extensively, making a financial donation to da.gd is encouraged.Other operations are outside the scope of this package.LinksCaptionLinkRepohttps://github.com/impredicative/dagdshort/Changeloghttps://github.com/impredicative/dagdshort/releasesPackagehttps://pypi.org/project/dagdshort/RequirementsPythonPython ≥3.9 is required. An older version of Python will not work.UsageTo install the package, run:$ pip install dagdshortUsage example:>>>importdagdshort# Setup>>>shortener=dagdshort.Shortener(user_agent_suffix='<YourGitUsername>/<YourGitRepoName>',max_cache_size=256)# Shorten>>>long_urls=['https://github.com/impredicative/dagdshort/','https://pypi.org/project/dagdshort/']>>>shortener.shorten_urls(long_urls){'https://github.com/impredicative/dagdshort/':'https://da.gd/W9s06','https://pypi.org/project/dagdshort/':'https://da.gd/KtGTB'}# Show cache info>>>shortener.cache_info{'Shortener._shorten_url':CacheInfo(hits=0,misses=2,maxsize=256,currsize=2)}To obtain the fastest response, URLs must be shortened together in a batch as in the examples above.
Up to 8 concurrent workers are automatically used. The max limit can, if really necessary, be changed by settingconfig.MAX_WORKERSbefore initializing the shortener.ErrorsIf a "Blacklisted long URL" error is experienced for a reasonable website which should not be blacklisted, it can be reportedhere. As an example, seethisissue.
Package: daget
🗃️ DaGetSimple utility to download datasets from data respositories.The goal of this project is to explore machine readable metadata and learn more about writing python packages.⚠️script is in early development and needs testing⚠️Installationto install daget using pip:pip install dagetUsageDownload dataset via DOI or landing page url:daget https://doi.org/10.5878/331q-3p13 ./destinationor short form doi:daget 10.5878/331q-3p13 ./destinationExample$ daget 10.7910/DVN/LFH4H8 ./test
destination:   /home/user/test
landing page:  https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/LFH4H8
 355.4KiB  Annex.pdf
 115.6KiB  List of items for measuring conceptions of democracy.csv
 81.1KiB  List of items for measuring conceptions of democracy.xlsx
 11.5KiB  Script - Part 1 Conceptions of democracy.R
 3.0KiB  Script - Part 2 Analyze items used to measure conceptions of democracy.R
 73.3KiB  Spreadsheet - Conceptions of democracy 10-2021.csv
 64.9KiB  Spreadsheet - Conceptions of democracy 10-2021.xlsx
 704.8KiB  downloadedSupported data respositories with file metadataschema.org/Datasethttps://dataverse.harvard.eduhttps://dataverse.nohttps://snd.se/cataloguefigsharehttps://su.figshare.comhttps://figshare.scilifelab.sezenodohttps://zenodo.orgAlternativesdatahugger- wider repository suportImprove the scriptAdding suport for additional repositories requires test cases and investigation arround how to get file metadata from the landing page.Please help by testing and reportingissuesTODOAdd error handlingCheck empty destination directoryImprove documentationPackage script for pip
Package: dag-factory
dag-factorydag-factoryis a library for dynamically generatingApache AirflowDAGs from YAML configuration files.InstallationUsageBenefitsContributingInstallationTo installdag-factoryrunpip install dag-factory. It requires Python 3.6.0+ and Apache Airflow 2.0+.UsageAfter installingdag-factoryin your Airflow environment, there are two steps to creating DAGs. First, we need to create a YAML configuration file. For example:example_dag1:default_args:owner:'example_owner'start_date:2018-01-01# or '2 days'end_date:2018-01-05retries:1retry_delay_sec:300schedule_interval:'03***'concurrency:1max_active_runs:1dagrun_timeout_sec:60default_view:'tree'# or 'graph', 'duration', 'gantt', 'landing_times'orientation:'LR'# or 'TB', 'RL', 'BT'description:'thisisanexampledag!'on_success_callback_name:print_helloon_success_callback_file:/usr/local/airflow/dags/print_hello.pyon_failure_callback_name:print_helloon_failure_callback_file:/usr/local/airflow/dags/print_hello.pytasks:task_1:operator:airflow.operators.bash_operator.BashOperatorbash_command:'echo1'task_2:operator:airflow.operators.bash_operator.BashOperatorbash_command:'echo2'dependencies:[task_1]task_3:operator:airflow.operators.bash_operator.BashOperatorbash_command:'echo3'dependencies:[task_1]Then in the DAGs folder in your Airflow environment you need to create a python file like this:fromairflowimportDAGimportdagfactorydag_factory=dagfactory.DagFactory("/path/to/dags/config_file.yml")dag_factory.clean_dags(globals())dag_factory.generate_dags(globals())And this DAG will be generated and ready to run in Airflow!If you have several configuration files you can import them like this:# 'airflow' word is required for the dagbag to parse this filefromdagfactoryimportload_yaml_dagsload_yaml_dags(globals_dict=globals(),suffix=['dag.yaml'])NotesHttpSensor (since 0.10.0)The packageairflow.sensors.http_sensorworks with all supported versions of Airflow. In Airflow 2.0+, the new package name can be used in the operator value:airflow.providers.http.sensors.httpThe following example showsresponse_checklogic in a python file:task_2:operator:airflow.sensors.http_sensor.HttpSensorhttp_conn_id:'test-http'method:'GET'response_check_name:check_sensorresponse_check_file:/path/to/example1/http_conn.pydependencies:[task_1]Theresponse_checklogic can also be provided as a lambda:task_2:operator:airflow.sensors.http_sensor.HttpSensorhttp_conn_id:'test-http'method:'GET'response_check_lambda:'lambdaresponse:"ok"inreponse.text'dependencies:[task_1]BenefitsConstruct DAGs without knowing PythonConstruct DAGs without learning Airflow primitivesAvoid duplicative codeEveryone loves YAML! ;)ContributingContributions are welcome! Just submit a Pull Request or Github Issue.
Package: dag-factory-hotfix
dag-factorydag-factoryis a library for dynamically generatingApache AirflowDAGs from YAML configuration files.InstallationUsageBenefitsContributingInstallationTo installdag-factoryrunpip install dag-factory. It requires Python 3.6.0+ and Apache Airflow 1.10+.UsageAfter installingdag-factoryin your Airflow environment, there are two steps to creating DAGs. First, we need to create a YAML configuration file. For example:example_dag1:default_args:owner:'example_owner'start_date:2018-01-01# or '2 days'end_date:2018-01-05retries:1retry_delay_sec:300schedule_interval:'03***'concurrency:1max_active_runs:1dagrun_timeout_sec:60default_view:'tree'# or 'graph', 'duration', 'gantt', 'landing_times'orientation:'LR'# or 'TB', 'RL', 'BT'description:'thisisanexampledag!'on_success_callback_name:print_helloon_success_callback_file:/usr/local/airflow/dags/print_hello.pyon_failure_callback_name:print_helloon_failure_callback_file:/usr/local/airflow/dags/print_hello.pytasks:task_1:operator:airflow.operators.bash_operator.BashOperatorbash_command:'echo1'task_2:operator:airflow.operators.bash_operator.BashOperatorbash_command:'echo2'dependencies:[task_1]task_3:operator:airflow.operators.bash_operator.BashOperatorbash_command:'echo3'dependencies:[task_1]Then in the DAGs folder in your Airflow environment you need to create a python file like this:fromairflowimportDAGimportdagfactorydag_factory=dagfactory.DagFactory("/path/to/dags/config_file.yml")dag_factory.clean_dags(globals())dag_factory.generate_dags(globals())And this DAG will be generated and ready to run in Airflow!NotesHttpSensor (since 0.10.0)The packageairflow.sensors.http_sensorworks with all supported versions of Airflow. In Airflow 2.0+, the new package name can be used in the operator value:airflow.providers.http.sensors.httpThe following example showsresponse_checklogic in a python file:task_2:operator:airflow.sensors.http_sensor.HttpSensorhttp_conn_id:'test-http'method:'GET'response_check_name:check_sensorresponse_check_file:/path/to/example1/http_conn.pydependencies:[task_1]Theresponse_checklogic can also be provided as a lambda:task_2:operator:airflow.sensors.http_sensor.HttpSensorhttp_conn_id:'test-http'method:'GET'response_check_lambda:'lambdaresponse:"ok"inreponse.text'dependencies:[task_1]BenefitsConstruct DAGs without knowing PythonConstruct DAGs without learning Airflow primitivesAvoid duplicative codeEveryone loves YAML! ;)ContributingContributions are welcome! Just submit a Pull Request or Github Issue.
Package: daggen
A Python port of thedaggentool proposed by Suter & Hunold.This library generates random, synthetic task graphs for simulation. You can easily convert the
generated task graphs into directed acyclic graphs (DAGs) in tools such asNetworkXorigraph, or export them to files. The library helps evaluate scheduling
algorithms using various application configurations.Here is an example of how to generate a DAG of 15 tasks, accepting the default parameters,
and import it into igraph:importdaggenasdgimportigraphasigdag=dg.DAG(seed=42,num_tasks=15)tasks,edges=dag.task_n_edge_dicts()igraph_dag=ig.Graph.DictList(vertices=tasks,edges=edges,directed=True)You can alternatively convert the task graph into a NetworkX’ DiGraph:importdaggenasdgimportnetworkxasnxdag=dag.DAG(seed=46,num_tasks=15)tasks,edges=dag.task_n_edge_tuples()net_dag=nx.DiGraph()net_dag.add_nodes_from(tasks)net_dag.add_edges_from(edges)Task and edge attributesEach generated task has two attributes:computation:a sequential cost (in Flops).alpha:the alpha parameter of Amdahl’s Law, used to encode the overhead of parallelizing tasks in parallel task graphs.Each edge represents a communication from a parent task to a child task and has adataattribute
representing the amount of data transferred from parent to child.DAG parametersYou can configure the characteristics of the generated DAGs by parametrizing theDAG()function.
Following the C implementation, one can set the following parameters:seed: used to seed the random number generator and ensure reproducibility.num_tasks: Number of computation nodes (application tasks) in the DAG.min_data: Minimum amount of data in bytes a task processes.max_data: Maximum amount of data in bytes a task processes.min_alpha: Minimum value for the extra parameter (e.g., Amdahl’s law parameter).max_alpha:  Minimum value for the extra parameter.fat: Width of the DAG, the maximum number of tasks executed concurrently. A small value results in a thin DAG.
(e.g., chain) with low task parallelism, while a large value creates a fat DAG (e.g., fork-join).
with a high degree of parallelism.density: Determines the dependencies between tasks of two consecutive DAG levels.regular: Regularity of the task distribution between the different levels of the DAG.ccr:  Communication to computation ratio. It encodes the complexity of the computation of a task
depending on the sizenof the dataset it processes. The encoding is as follows:1 :a * n(ais a constant picked randomly between 26 and 29).2 :a * n log n3 :n3/20 : Random choice among the three complexities.jump_size:   Maximum number of levels spanned by inter-task communications, which enables DAGs
with execution paths of multiple lengths.Examples on Google ColabA couple examples are available on this Colab notebook:
Package: dagger-io
Dagger Python SDKA client package for runningDaggerpipelines.What is the Dagger Python SDK?The Dagger Python SDK contains everything you need to develop CI/CD pipelines in Python, and run them on any OCI-compatible container runtime.RequirementsPython 3.10 or laterDocker, or another OCI-compatible container runtimeA compatible version of theDagger CLIis automatically downloaded and run by the SDK for you, although it’s possible to manage it manually.InstallationFromPyPI, usingpip:pipinstalldagger-ioYou can also install viaConda, from theconda-forgechannel:condainstalldagger-ioExampleCreate amain.pyfile:importsysimportanyioimportdaggerfromdaggerimportdagasyncdefmain(args:list[str]):asyncwithdagger.connection():# build container with cowsay entrypointctr=(dag.container().from_("python:alpine").with_exec(["pip","install","cowsay"]).with_entrypoint(["cowsay"]))# run cowsay with requested messageresult=awaitctr.with_exec(args).stdout()print(result)anyio.run(main,sys.argv[1:])Run with:$pythonmain.py"Simple is better than complex"_____________________________| Simple is better than complex |=============================\\^__^(oo)\_______(__)\       )\/\||----w |||     ||NoteIt may take a while for it to finish, especially on first run with cold cache.If you need to debug, you can stream the logs from the engine with thelog_outputconfig:config=dagger.Config(log_output=sys.stderr)asyncwithdagger.connection(config):...Learn moreDocumentationAPI ReferenceSource codeDevelopmentThe SDK is managed with a Dagger module in./dev. To see which tasks are
available run:daggercall-mdevCommon tasksRun pytest in supported Python versions:daggercall-mdevtestsCheck for linting violations:daggercall-mdevlintcheckRe-format code following common styling conventions:daggercall-mdevlintformat-o.Update pinned devevelopment dependencies:daggercall-mdevlock-o.Build and preview the reference documentation:daggercall-mdevdocspreviewupAdd--helpto any command to check all the available options.Engine changesTesting and regenerating the client may fail if there’s changes in the engine code that haven’t been released yet. Prefix withhack/devto build a new engine before executing pipelines:../../hack/devdaggercall-mdevtestTo re-generate the client (codegen) after changes to the API schema:./hack/dev./hack/makesdk:python:generate
Package: daggerml
Dagger-ML Python LibraryPrerequisitespipxIfpipxis not installed, first do that.hatchThen installhatchvia:pipx install hatch.Configuration# helppython-mdaggerml--help# configure global API endpointpython-mdaggermlconfigure\--global\--profile${DML_PROFILE}\# optional--api-endpointhttps://api.${DML_ZONE}-${AWS_REGION}.${DOMAIN}# configure API keypython-mdaggermllogin\--profile${DML_PROFILE}\# optional--username${USERNAME}# configure group ID for local projectpython-mdaggermlconfigure--group-id${DML_GROUP_ID}# bash command completionpipinstallargcomplete# enable completion for dml in this shelleval"$(register-python-argcompletedml)"# enable completion for dml via bashrccat<<'EOT' >> ~/.bashrceval "$(register-python-argcomplete dml)"EOT# or see: https://kislyuk.github.io/argcomplete/#activating-global-completionUsageYou currently needAWS_DEFAULT_REGIONandDML_ZONEenvironment variables
set. Then you can runpython bootstrap-docker.py, for instance.bootstrap-docker.pysets up the docker-build func, so you can now run docker
stuff in your dags (e.g. as we do in the docs/examples/ directory).Run Locally# Start local postgres:sudosystemctlstartpostgresql# Connect to local postgres:psql-hlocalhostpostgrespostgres# Start local DML API server:pythoninfra/lib/api/server.py# Run dag locallyDML_LOCAL_DB=1pythonmydag.pyDocsTo build the docs, first make surebootstrap-docker.pyhas been run, then
run:hatch run docs:buildTo serve the docs:hatch run docs:serveTestsTo run the tests:hatch run test:cov