Saravanan666/pypi-org huggingface dataset:
Package: migperf
MIG ProfilerMIGProfiler is a toolkit for benchmark study on NVIDIAMIGtechniques. It provides profiling on multiple deep learning training and inference tasks on MIG GPUs.MIGProfiler is featured for:🎨 Support a lot of deep learning tasks and open-sourced models on a various of benchmark type📈 Presentcomprehensivebenchmark results🐣Easy to usewith a configuration file (WIP)The project is under rapid development! Please check ourbenchmark websiteand join us!Benchmark WebsiteInstallQuick StartCite UsContributorsAcknowledgementLicenseBenchmark Website 📈Coming soon!Install 📦️Manual installRequirements:PyTorch with CUDAOpenCVSanicTransformersTqdmPrometheus client# create virtual environmentcondacreate-nmig-perfpython=3.8
condaactivatemig-perf# install required packagescondainstallpytorchtorchvisionpytorch-cuda=11.6-cpytorch-cnvidia
condainstall-cconda-forgeopencv
pipinstalltransformers
pipinstallsanictqdmprometheus_clientPyPI installWIPUse DockerWIPQuick Start 🚚You can easily to profile on MIG GPU. Below are some common deep learning tasks to play with.1. MIG training benchmarkWe first create a1g.10gbMIG device# enable MIGsudonvidia-smi-i0-mig1# create MIG instancesudonvidia-smimig-cgi1g.10gb-CStart DCGM metric exporterdockerrun-d--rm--gpusall--netmig_perf-p9400:9400\-v"${PWD}/mig_perf/profiler/client/dcp-metrics-included.csv:/etc/dcgm-exporter/customized.csv"\--namedcgm_exporter--cap-addSYS_ADMINnvcr.io/nvidia/k8s/dcgm-exporter:2.4.7-2.6.11-ubuntu20.04\-c500-f/etc/dcgm-exporter/customized.csv-dfStart to profilecdmig_perf/profilerexportPYTHONPATH=$PWDpythontrain/train_cv.py--bs=32--model=resnet50--num_batches=500--mig-device-id=0Remeber to disable MIG after finish benchmarksudonvidia-smi-i0-dci
sudonvidia-smi-i0-dgi
sudonvidia-smi-i0-mig02. MIG inference benchmarkStart DCGM metric exporterdockerrun-d--rm--gpusall--netmig_perf-p9400:9400\-v"${PWD}/mig_perf/profiler/client/dcp-metrics-included.csv:/etc/dcgm-exporter/customized.csv"\--namedcgm_exporter--cap-addSYS_ADMINnvcr.io/nvidia/k8s/dcgm-exporter:2.4.7-2.6.11-ubuntu20.04\-c500-f/etc/dcgm-exporter/customized.csv-dfStart to profilecdmig_perf/profilerexportPYTHONPATH=$PWDpythonclient/block_infernece_cv.py--bs=32--model=resnet50--num_batches=500--mig-device-id=0See more benchmark experiments in./exp.3. Visualizein notebookin Prometheus (under improvement)Cite Us 🌱@article{zhang2022migperf,title={MIGPerf: A Comprehensive Benchmark for Deep Learning Training and Inference Workloads on Multi-Instance GPUs},author={Zhang, Huaizheng and Li, Yuanming and Xiao, Wencong and Huang, Yizheng and Di, Xing and Yin, Jianxiong and See, Simon and Luo, Yong and Lau, Chiew Tong and You, Yang},journal={arXiv preprint arXiv:2301.00407},year={2023}}Contributors 👥Yuanming LiHuaizheng ZhangYizheng HuangXing DiAckowledgementSpecial thanks to Aliyun and NVIDIA AI Tech Center to provide MIG GPU server for benchmarking.LicenseThis repository is open-sourced underMIT License.
Package: migra
migra: Like diff but for Postgres schemascompare schemasautogenerate migration scriptsautosync your development database from your application modelsmake your schema changes testable, robust, and (mostly) automaticmigrais a schema diff tool for PostgreSQL, written in Python. Use it in your python scripts, or from the command line like this:$ migra postgresql:///a postgresql:///b
alter table "public"."products" add column newcolumn text;

alter table "public"."products" add constraint "x" CHECK ((price > (0)::numeric));migramagically figures out all the statements required to get from A to B.Most features of PostgreSQL are supported.Migra supports PostgreSQL >= 9 only.Known issues exist with earlier versions. More recent versions are more comprehensively tested. Development resources are limited, and feature support rather than backwards compatibility is prioritised.THE DOCSDocumentation is atdatabaseci.com/docs/migra.Folks, schemas are goodSchema migrations are without doubt the most cumbersome and annoying part of working with SQL databases. So much so that some people think that schemas themselves are bad!But schemas are actually good. Enforcing data consistency and structure is a good thing. It’s the migration tooling that is bad, because it’s harder to use than it should be.migrais an attempt to change that, and make migrations easy, safe, and reliable instead of something to dread.ContributingContributing is easy.Jump into the issues, find a feature or fix you'd like to work on, and get involved. Or create a new issue and suggest something completely different. If you're unsure about any aspect of the process, just ask.Creditsdjrobstep: initial development, maintenancealvarogzp: privileges supportseblucas: docker improvementsMOZGIII: docker supportmshahbazi: misc fixes and enhancements
Package: migra-idempotent
migra: Like diff but for Postgres schemascompare schemasautogenerate migration scriptsautosync your development database from your application modelsmake your schema changes testable, robust, and (mostly) automaticmigrais a schema diff tool for PostgreSQL, written in Python. Use it in your python scripts, or from the command line like this:$ migra postgresql:///a postgresql:///b
alter table "public"."products" add column newcolumn text;

alter table "public"."products" add constraint "x" CHECK ((price > (0)::numeric));migramagically figures out all the statements required to get from A to B.Most features of PostgreSQL are supported.Migra supports PostgreSQL >= 9 only.Known issues exist with earlier versions. More recent versions are more comprehensively tested. Development resources are limited, and feature support rather than backwards compatibility is prioritised.THE DOCSDocumentation is atdatabaseci.com/docs/migra.Folks, schemas are goodSchema migrations are without doubt the most cumbersome and annoying part of working with SQL databases. So much so that some people think that schemas themselves are bad!But schemas are actually good. Enforcing data consistency and structure is a good thing. It’s the migration tooling that is bad, because it’s harder to use than it should be.migrais an attempt to change that, and make migrations easy, safe, and reliable instead of something to dread.ContributingContributing is easy.Jump into the issues, find a feature or fix you'd like to work on, and get involved. Or create a new issue and suggest something completely different. If you're unsure about any aspect of the process, just ask.Creditsdjrobstep: initial development, maintenancealvarogzp: privileges supportseblucas: docker improvementsMOZGIII: docker supportmshahbazi: misc fixes and enhancements
Package: migranite
Tool for manage migrations in your project.Installpip install migraniteIf you want to use MongoDB as database for migrations:pip install 'migranite[mongo]'How to startTo initialize the migranite in your project run:migranite init --migrations migrations_dir --templates templates_dirThis command creates the config file (.migranite) and two directories (migrations_dirandtemplates_dir).Create the first migration:migranite create my-first-migrationThis is create the filemigrations_dir/001-my-first-migration.pywith an empty migration.
You need to write a module docstring and implement functionrun()with your migration logic.Show the available migrations with statuses:migranite listRun all migrations that haven’t been started earlier:migranite runRun only specified migrations:migranite run my-first-migrationRun the migrations that were started earlier:migranite run --force my-first-migrationConfig fileA config file is a simple ini-file with three sections.[migrations]Migrations settings.path:Path to directory with migrations.digits:Number of digits in migration number. Default3.[templates]Templates settings.path:Path to directory with templates.default:Default template file name.[database]Database settings. Currently only JSON file and MongoDB is supported.backend:Type of database backend (jsonormongo).Other settings are backend-specified.jsonpath:path to json file.mongohost:Hostname or ip address. Defaultlocalhost.port:Default27017.name:Name of database. Required.collection:Name of collection. Defaultmigrations.
Package: migrant
Migrant is a database schema version management frameworkFeatures include:backend agnostic coreexplicit migration script orderingsupport for downgradingsupport for out-of-order migrationssupport for migrating multiple homogenuous databasesDevelopmentTo set up development environment, usepipenv:pipenv install --devTo run tests, usepytest:pytestTo run tests under all supported environments, usetox:tox --skip-missing-interpretersTo check for typing errors, usemypy:mypy srcCHANGELOG1.5.0 (2023-01-24)License under MIT1.4.0 (2020-04-28)Ability to skip database from migration, when it not available.API change: to support parallel processing Backend class grew connection
management methods, that needs to be implemented in plugins.Type annotations addedSupport migration of multiple databases in parallel (see–parallelcommand line argument for  command)Add support for Python-3.8Drop support for Python-2.7Modernize development environment with pipenv1.3.1 (2019-03-01)Updated the script template to make it lint friendly.1.3.0 (2018-07-11)Suppport for Python 3.6 and 3.7. Dropped Python 3.5.Fixed compatibility to not usefuture, since it importedConfigParserasconfigparserin Python 2.7. Instead the officialconfigparserbackport
is used.1.2.1 (2018-02-06)Bugfix: proper error whencmdargument missing in Python 31.2.0 (2018-02-06)Python 3 compatibility.1.1.1 (2017-05-23)Update README.rst badges.1.1.0 (2017-05-23)First public release using all public tools.1.0.0 (2017-05-16)First packaged version.
Package: migranto
# MigrantoSimple SQL migration tool for SQLite and PostgreSQL## Instalationpip install migranto## MigrationsAll migrations exist in one directory with names like:0001_some_useful_name.sql0001_some_useful_name_rollback.sql0002_some_other_useful_name.sql0002_some_other_useful_name_rollback.sqlMigranto checks migrations directory and apply sql files to database.That's it. Simple enough.## Usage:`./migranto [-h] --database URL [--storage NAME] [--name NAME] `shows migrations status`./migranto [-h] --database URL --name NAME --path PATH [--migration N] [--storage NAME] [--verbose] [--fake] [--out] `*	__--help, -h__ — show this help message and exit*	__--database URL, -d URL__ — database url (pgsql://user:password@host:port/dbname or sqlite://path/to/data.base)*	__--name NAME, -n NAME__ — migration set name*	__--path PATH, -p PATH__ — path to migrations dir*	__--migration N, -m N__ — migration step (last migration if omitted, 0 - before first migration)*	__--storage NAME, -s NAME__ — migranto table name for data (default is migranto)*	__--verbose, -v__ — verbose mode*	__--out, -o__ — output sql to standard output. nothing applied to database.*	__--fake, -f__ — fake migration (no sql applied to database, except migration number)
Package: migrantpy
# Migrantpy
[![License](http://img.shields.io/:license-apache-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)Simple SQLite database migrations library for Python.## UsageThe library can be used either as a command line utility:`bash migrantpy migrate database.sqlite migrations_folder/ `Or as aPythonlibrary:`python import migrantpymigrantpy.migrate('database.sqlite','migrations_folder/')`## Hacking
You can create a full test database this way:`bash ./migrant.py migrate test.sqlite test_migrations `Running the unit tests:`bash python3-munittest discover--locals`Or install locally:`bash python setup.py install--usermigrate-h`## Download
Install the library from the Python Package Index:`bash pip install migrantpy `## LicenseCopyright 2016 Sascha PeilickeLicensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Package: migrate-code
migrate-codeTable of Contentsmigrate-codeLinksInstallationQuickstartLicenseLinksGitLabGitHubInstallationpip install migrate-codeQuickstartMigration stages will be converted into commits when the upgrade is applied. Themigration.pyfile should define aMigrationvariable either calledmormigration.# migration.py

from migrate_code import Migration, get_repo_root

m = Migration("Demonstrate basic usage")

@m.add_stage(1, "Create a new file")
def stage_1():
    (get_repo_root() / "new_file.txt").write_text("Hello world")


@m.add_stage(2, "Create another file")
def stage_2():
    (get_repo_root() / "another_file.txt").write_text("Hello another world")


@m.add_stage("1.1", "Modify the first file")
def stage_1_1():
    (get_repo_root() / "new_file.txt").write_text("Hello world, again")Now it is easy to create and reset commits:$gitinit$gitaddmigration.py$gitcommit-m"Add migration.py"$migrate-codelog------- (2) Create another file------- (1.1) Modify the first file------- (1) Create a new file$migrate-codecurrentNone$migrate-codeupgradeRunning migration 1: Create a new fileRunning migration 1.1: Modify the first fileRunning migration 2: Create another file$gitlog--oneline5041191 (HEAD -> master) Create another file92012af Modify the first filed4d0611 Create a new file78438e2 Add migration.py$migrate-codelog* 5041191 (2) Create another file92012af (1.1) Modify the first filed4d0611 (1) Create a new file$migrate-codecurrent2$migrate-coderesetResetting migration 2: Create another fileResetting migration 1.1: Modify the first fileResetting migration 1: Create a new file$gitlog--oneline78438e2 (HEAD -> master) Add migration.py$migrate-codelog------- (2) Create another file------- (1.1) Modify the first file------- (1) Create a new file$migrate-codeupgrade--stage1.1Running migration 1: Create a new fileRunning migration 1.1: Modify the first file$gitlog--oneline7b65210 (HEAD -> master) Modify the first filea7c1027 Create a new file78438e2 Add migration.py$migrate-codelog------- (2) Create another file* 7b65210 (1.1) Modify the first filea7c1027 (1) Create a new fileLicensemigrate-codeis distributed under the terms of theMITlicense.
Package: migrate-redmine-to-gitlab
# Migrate redmine to GitLab[![Build Status](https://travis-ci.org/ultreia-io/migrate-redmine-to-gitlab.svg?branch=master)](https://travis-ci.org/ultreia-io/migrate-redmine-to-gitlab) [![PyPI version](https://badge.fury.io/py/migrate-redmine-to-gitlab.svg)](https://badge.fury.io/py/migrate-redmine-to-gitlab)Migrate projects from Redmine to Gitlab.Documentation can be found [here](https://ultreia-io.github.io/migrate-redmine-to-gitlab).Enjoy.Tony.
Package: migrations
Simple, database-agnostic migration tool for Python applications.
Inspired bynode migrations.StatusThe project is in beta now. Bugs and breaking changes may occur.FeaturesNo specific database requirements, use it for anything you call database.Pretty simple, just generate migration script and define self-explanatoryup()anddown()functions there.Use imports in your migration scripts to load database bindings. Talk to the
database the same way your application does.Stores the sequence of already performed migrations. If the sequence does not
match scripts in migrations directory (e.g. after merge), aborts and warns user.TBD: Deeply configurable, including resources acquiring and releasing.RequirementsOnly Python 3 is supported for now.Installation$pipinstallmigrationsNotice, this distribution provides package and executable
script namedmigrate, so check if it does not mess with
existing packages/scripts. Generally, you should neither install
this tool globally, nor install several migration tools for one project.Usageusage: migrate [options] [action]

actions:
  up     [-h] [NAME|COUNT]   (default) perform COUNT migrations or till
                             given NAME (by default perform all available)
  down   [-h] [NAME|COUNT]   revert COUNT migrations or till
                             given NAME (by default revert one)
  create [-h]  NAME          create new migration file

  show   [-h]                print all migrations in chronological order

options:
  -h, --help                 show this help message and exit
  -v, --version              show version and exit
  -d PATH, --migrations-dir PATH
                             directory where migrations are stored
  -s PATH, --state-file PATH
                             location of file which stores database state
  -t PATH, --template-file PATH
                             location of template file for new migrationsEach migration file must define functionsup()anddown()without required arguments.Simple migration example:importredisdb=redis.Redis(host='localhost',port=6379)defup():db.rpush('used_libraries','migrations')defdown():db.rpop('used_libraries','migrations')Current working directory is prepended tosys.path, so anyimportstatement in migration file tries to find requested
module in working directory at first. You can use this to manage
database access for your both app and migrations with single piece
of code. See an example. Let’s assume that in working directory
we have module nameddb, which contains singleton object
responsible for DB connection, for examplePyMySQLConnection object.fromdbimportconnectiondefmanage_cursor(action):defwrap():withconnection.cursor()ascursor:action(cursor)connection.commit()returnwrap@manage_cursordefup(cursor):cursor.execute("INSERT INTO used_libraries (`name`) VALUES ('migrations')")@manage_cursordefdown(cursor):cursor.execute("DELETE FROM used_libraries WHERE `name`='migrations'")
Package: migration-toolkit
Flywheel Migration Toolkitflywheel-migrationis a library that provides configurable parsing and
de-identification of foreign data.DevelopmentInstall the project usingpoetryand enablepre-commit.poetryinstall
pre-commitinstallTestingTests are handled in pre-commit containers usingqa-ci.pre-commitrun-apytestTo run all pre-commit hooks, run:pre-commitrun-aThis will execute all hooks defined in.pre-commit-config.yaml.Alternatively, you can run pytests natively by executing the poetry virtual environment,
then callingpytestwithpoetryshell
pytest-vvtests/To run individual tests natively such astest_jitter_for_dicomintest_dicom_file_profile.pypoetryshell
pytest-vvtests/test_dicom_file_profile.py-sktest_jitter_for_dicomReleaseReleases are handled with qa-ci automation. Head toBuild >> Pipelinespage on repository's sidebar, then click onRun Pipeline.
SelectVariableinVariablesdropdown, and entereREASEASEasvariable key.
Setvariable valueto desired version and click onRun Pipeline. This will
create a new release MR that can than be reviewed and merged to deploy a new release.DocumentationDocumentation is automatically handled bypre-commithookgenerate_docs. The
HTML page is built intopublicand deployed to GitLab pages. Documentation is
available at Documentation athttps://flywheel-io.gitlab.io/public/migration-toolkit/.
Package: migratore
Simple migration framework / infra-structure for SQL based databases.InstallationpipinstallmigratoreExecutionHOST=${HOST}DB=${DB_NAME}USERNAME=${DB_USER}PASSWORD=${DB_PASS}migratoreupgradeVariablesHOST- Hostname or IP address of the database system for migrationPORT- TCP port to be used in the connection with the database systemUNIX_SOCKET- Filesystem path to the UNIX socket file to be used in connectionDB- Name of the database used as the migration targetUSERNAME- Username for authentication in databasePASSWORD- Password to be used for authentication in databaseFS- Base file system path for file migration (may depend on migration context)Commandshelp- Prints a help message about the CLI interfaceversion- Prints the current version of migratoreenviron- Displays the current environment in the standard outputlist- Lists the executed migrations on the current databaseerrors- Lists the various errors from migration of the databasetrace [id]- Prints the traceback for the error execution with the provided idrebuild [id]- Run the partial execution of the migration with the given idupgrade [path]- Executes the pending migrations using the defined directory or currentskip [path]- Skips the current migration (next to be run) using the defined directory or currentgenerate [path]- Generates a new migration file into the target pathExamplesdatabase=Migratore.get_database()table=database.get_table("users")table.add_column("username",type="text")LicenseMigratore is currently licensed under theApache License, Version 2.0.Build Automation
Package: migratron
MigratronA database migration system to modify the database schema or to populate the tables
information.This will work with:PostgreSQLHivePrestoSQL/PrestoDBChange LogAll notable changes to this project will be documented here.Sort subsections like so: Added, Bugfixes, Improvements, Technical tasks.
Group anything an end user shouldn’t care deeply about into technical
tasks, even if they’re technically bugs. Only include as “bugfixes”
bugs with user-visible outcomes.When major components get significant changes worthy of mention, they
can be described in a Major section.v2.1.0 - 2020-11-17AddedOfficial docker imageBugfixesDocument what migratron doesException message is valid when using Python2 and Python3Make the subcommand required and print the help message if missingTechnical TasksImprove the documentation on how to setup migratron locallyAdd documentation on how to run the testsAdd documentation on how migratron worksv2.0.1 - 2020-09-29BugfixesAdded VERSION file to the python packagev2.0.0 - 2020-08-28AddedOptions to run the migration against Hive or PrestoCreated the documentationThe project can be used on Python3Description to the README.rstChangedRename the package to migratronChanged the--database-uri|typeparameters to--db-uri|typeBugfixesAdded requirements-dev.txt to setup.py package_dataMissing requirements.txt when using pip to install packageUse Python 2 and 3 compatible function to get user inputTechnical TaskDo not use unittest deprecated methodsGenerate coverage report when running the testsAdded missing classifiers to setup.pyCreate a version fileFormat the code using blackAdd Makefile to run tests and generate documentationSolve typos and improve documentationFix min required version for some dev dependenciesAdded readthedocs configuration fileAdded long description to the setup.pyUse MANIFEST.ini to add non python files to the package<Paste>