Saravanan666/pypi-org huggingface dataset:
Package: mlfields
MLFields: ML input managementWhy I develop thisI feel it is hard to manage features in ML projects;During experiment phase, I often do many experiments and lose track of feature historyDuring oepration phase, I need to build monitoring system each timeThe solution I want to build in this librarySystem manages features in following structureProject: unit of managing featuresSet of feature definitionsGenerated featuresExperiments (evaluation of generated features)
Package: mlfix
mlfixMotivationCurrently the is no way to natively make models trained using mlflow portable across local machines or from remote to local,
while keeping all the benefits (working commands etc.) of the mlflow environment (see for examplethis issue).This is especially bad in small teams, local development or just prototyping.This tool makes it easy to fix existing mlflow artifact store to current path.Future work may include migrating existing artifact stores, only specific experiments etc.Currentlymlf-corealso supports such functionality, but if you are not using mlf-core and want just to fix your mlruns, this tiny tool will help you.InstallationThis is tested for Python 3.6 to 3.9.FromPyPI:$pipinstallmlfixFrom the source code (in the main directory):$python-mpipinstall.Usage$mlfixpath_to_artifact_storeThat is it!You must specify the name of themlrunsfolder if it was different than the default in the former location of the store:$mlfix--mlruns-namenonstandard_mlrunspath_to_artifact_store
Package: mlflow-aws-rds-iam
MLflow AWS RDS IAM Auth PluginsMLflow plugins adding AWS RDS IAM authentication for tracking store and model registry store.RequirementsPython 3.10+MLflow 2+Boto3Only Postgres and MySQL are supported (the AWS RDS database engines supported by MLflow as remote backend store).Installationpip install mlflow-aws-rds-iamGetting StartedThe plugins are mainly intented for the use of MLflow in server mode, allowingAWS RDS IAM authenticationfor the backend store.mlflow server --backend-store-uri <database URI> --artifacts-destination <artifact store URI>ConfigurationIf apasswordis provided in the database URI, AWS IAM authentication will be skipped,effectively disabling the plugins.Credentialsare required in order to use AWS IAM authentication. Refer toBoto3 documentationfor configuration instructions.SSL connection to the database is enabled by default, but can be disabled with the environment variableMLFLOW_DISABLE_DB_SSL=true.Debug log of the plugins and underlying libraries (boto3, urllib3) can be enabled withMLFLOW_AWS_RDS_IAM_DEBUG=true.LicenseThis project is licensed under the terms of the MIT license.AyzrFree and Open Source project.
Package: mlflow-baidubce-store-plugin
mlflow-baidubce-store-pluginAMLflowpluginthat allows users to useBaiduBCE BOS(CN/EN) as the artifact store for MLflow.Examplepipinstallmlflow-baidubce-store-plugin-Uimportmlflow.pyfuncimportosos.environ["MLFLOW_BOS_ENDPOINT"]="bj.bcebos.com"os.environ["MLFLOW_BOS_SECRET_ACCESS_KEY"]="AK"os.environ["MLFLOW_BOS_KEY_ID"]="SK"classMod(mlflow.pyfunc.PythonModel):defpredict(self,ctx,inp):return8765exp_name="bos-exp"mlflow.create_experiment(exp_name,artifact_location="bos://mlflow-test/")mlflow.set_experiment(exp_name)mlflow.pyfunc.log_model('model_test',python_model=Mod())print(mlflow.get_artifact_uri())mlflow.artifacts.download_artifacts(mlflow.get_artifact_uri())
Package: ml-flow-client
ML-Flow-ClientThe simple insurance model building tool for rapid development and deployment.DescriptionML-Flow-Client is a platform to simplefy machine learning development ofr insurance companies.QuickstartInstall using pipInstall MLflow from PyPI viaml-flow-clientDocumentationOfficial documentation for Ml-Flow-Clien can be found athttps://ml-flow-py-client.readthedocs.io/en/latest/NoteThis project has been set up using PyScaffold 3.3.1. For details and usage
information on PyScaffold seehttps://pyscaffold.org/.
Package: mlflow-cratedb
MLflow adapter for CrateDBAboutAn adapter forMLflowto useCrateDBas a storage database forMLflow
Tracking. MLflow is an open source platform to manage the whole ML lifecycle,
including experimentation, reproducibility, deployment, and a central model
registry.SetupInstall the most recent version of themlflow-cratedbpackage.pipinstall--upgrade'mlflow-cratedb[examples]'To verify if the installation worked, you can inspect the version numbers
of the software components you just installed.mlflow-cratedb--version
mlflow-cratedbcratedb--versionDocumentationTheMLflow Trackingsubsystem is about recording and querying experiments, across
code, data, config, and results.The MLflow adapter for CrateDB can be used in different ways. Please refer
to thehandbook, the documentation aboutcontainer usage, and thehands-on guidelines.DevelopmentFor joining the development, or for making changes to the software, read about
how toinstall a development sandbox.Project InformationResourcesSource codeDocumentationPython Package Index (PyPI)ContributionsThis library is an open source project, and ismanaged on GitHub.
Every kind of contribution, feedback, or patch, is much welcome.Create an
issueor submit a patch if you think we should include a new feature, or to
report or fix a bug.DevelopmentIn order to set up a development environment on your workstation, please head
over to thedevelopment sandboxdocumentation. When you see the software
tests succeed, you should be ready to start hacking.LicenseThe project is licensed under the terms of the Apache License 2.0, likeMLflowandCrateDB, seeLICENSE.AcknowledgementsSiddharth Murching,Corey Zumar,Harutaka Kawamura,Ben Wilson, and
all other contributors for conceiving and maintainingMLflow.Andreas Niggfor contributing thetracking_merlion.pyandtracking_pycaret.pyML experiment programs, usingMerlionandPyCaret.
Package: mlflow-databricks-artifacts
mlflow-databricks-artifactsAn artifact repository plugin for MLflow that extends ACL'ed artifacts support to older MLflow versionsVersion 2.0Backport SSE-CPK support to older version Mlflow.Related PRhttps://github.com/mlflow/mlflow/pull/6927/filesDochttps://docs.google.com/document/d/1EXrv4-bYoAWl5vRQ1ksRgwqUjheNYgIgeKhfLZQOtRQ/edit#The mlflow-databricks-artifacts is based on mlflow version v1.29.0
Package: mlflow-dbstore
SQL Server plugin for MLflowThis repository provides a MLflow plugin that allows users to use SQL Server as the artifact store for MLflow.Implementation overviewThis repository contains one Python package:dbstoreplugin: This package includes theDBArtifactRepositoryclass that is used to read and write artifacts from SQL databases. This class sets the attributeis_plugin = Truein order to indicate that the class is an MLflow artifact repository plugin. This package also includes the SQLAlchemy database models referenced byDBArtifactRepository. The package'ssetup.pyfile defines entrypoints that tell MLflow to automatically associate themssqlURIs with theDBArtifactRepositoryimplementation when thedbstorepluginlibrary is installed. The entrypoints are configured as follows:entry_points={
    "mlflow.artifact_repository": [
      "mssql=sqlplugin.store:DBArtifactRepository"
    ]
},User experienceThe proposed plugin structure and development workflow provide the following experience to the end user:Users can simply install MLflow with the SQL Server plugin viapip install mlflow[sqlserver]and then use MLflow as normal. The SQL Server artifact support will be provided automatically using the previously-described setup entrypoints mechanism.
Package: mlflow-deploifai
MLFLow plugin for DeploifaiWhat is Deploifai?Deploifai is a modern developer platform for Machine Learning. We build
tools to automate cloud infrastructure, and make the best tools for data
science and ML simple and straightforward for the community.What does this plugin do?This plugin helps the MLFlow client to connect and work with the Deploifai
MLFlow community server. Without this plugin, Deploifai MLFlow server cannot be
used.How to installpipinstallmlflow-deploifaiOnce the plugin is installed, use MLFlow just as you would normally.Usageimportmlflowmlflow.set_tracking_uri('https://community.mlflow.deploif.ai')mlflow.set_experiment("<workspace>/<project>/<experiment>")ContributingWe are working on a contribution page. We welcome you raising any issues,
and even having your PR submissions. Thanks!
Package: mlflow-deploy-cli
mlflow-deploymentCLI for deployment ofMLFlowwith support for backend and artifact stores.Deploy MLFlow with support for PostgreSQL, Azure Blobs, Amazon S3 and Google Cloud Platform with ease.Usagepip install mlflow-deploy-clipython -m mlflow_deploy_cli --artifact-store '<desired_store>'can beazure,s3,gcporlocalcd builddocker build . -t mlflowdocker-compose up -f docker-compose.yml upMLFlow is up and running 🚀.
Package: mlflow-dl
MLFlow model downloaderAllows to download models fromMLFlowModel Registry using model's status (Staging or Production) or model's Version.NOTES:supports MLflow's version <1.20.0tested only with Tensorflow/Keras modelstested on S3 and SFTP storagesInstallationInstall from PyPipipinstallmlflow-dlInstall from GitHubgitclonegit@github.com:dem-artem/mlflow_dl.gitcdmlflow_dlfor production:pipinstall.for development:cdmlflow_dl
pipinstall-e".[dev]"Environment Variables ConfigurationIn the most of the cases you need to configure a proper credentials to mlflow and related storage.
Built in variables:ENV NameDescriptionMLFLOWDL_TARGET_FOLDER_LOCALThe folder where downloaded results placedAlso, you may need to configure variables for access to MLFlow and AWS S3 bucket. Some of them:ENV NameDescriptionAWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYAWS_DEFAULT_REGIONAWS_BUCKET_NAMEMLFLOW_TRACKING_USERNAMEUsername for HTTP Base Auth for MLFlow (if used)MLFLOW_TRACKING_PASSWORDPassword for HTTP Base Auth for MLFlow (if used)MLFLOW_TRACKING_URIURL to MLFlow UI (required)Usage from Command LineGet list of available commands:python-mmlflow_dlormlflow_dlDownload models in "Production" stage:mlflow_dldownload-production-mmodel_name_1,model_name_2Download models in "Staging" stage:mlflow_dldownload-staging-mmodel_name_1,model_name_2Download model by name and version number:mlflow_dldownload-model-by-version-mmodel_name-v1Download specific folder from the root of model's experiment:mlflow_dldownload-folder-by-model-version-mmodel_name-v1-fcustom_folderDownload specific folder forTensorflow Serving compatible structure:mlflow_dldownload-folder-by-model-version-mmodel_name-v1-fcustom_folder--no-subfolderUsage from Codefrommlflow_dlimportMlflowDlmlflowdl=MlflowDl(target_folder="tmp_folder")mlflowdl.download_model_by_version("model_name","1")DevelopmentTestingpython-munittestCoveragepython-mcoveragerun-munittest
python-mcoveragehtml-iCode Stylepython-mflake8VersioningCommit your changes and run with a proper label (major | minor | patch):bumpversionpatch
gitpushorigin
gitpushorigin--tagsBuild packagerm-rf./dist
python-mbuildUpload to custom PyPIIn case of private repo you need to make sure you configured credentials in~/.pypirc, than run:twineupload--repository{repo_name}dist/*
Package: mlflow-elasticsearchstore
mlflow-elasticsearchstoreMlflow plugin to use ElasticSearch as backend for MLflow tracking service. To use this plugin you need a running instance of Elasticsearch 6.X.Run 'pip install mlflow-elasticsearchstore' to register the plugin as an entrypoint with Elasticsearch backend.$pipinstallmlflow-elasticsearchstoreDevelopmentIn a python environment (you can use the one where mlflow is already installed):$gitclonegitclonehttps://github.com/criteo/mlflow-elasticsearchstore.git
$cdmlflow-elasticsearch
$pipinstall.How Tomlflow-elasticsearchstore can now be used with the "elasticsearch" scheme, in the same python environment :$mlflowserver--host$MLFLOW_HOST--backend-store-urielasticsearch://$USER:$PASSWORD@$ELASTICSEARCH_HOST:$ELASTICSEARCH_PORT--port$MLFLOW_PORT--default-artifact-root$ARTIFACT_LOCATION
Package: mlflow-extend
MLflow ExtendExtend MLflow's functionality.InstallationFrom PyPIpipinstallmlflow-extendFrom GitHub (development version)pip install git+https://github.com/harupy/mlflow-extend.gitExampleimportnumpyasnpimportpandasaspdimportmatplotlib.pyplotaspltfromplotlyimportgraph_objectsasgofrommlflow_extendimportmlflowwithmlflow.start_run():# mlflow native APIsmlflow.log_param('param',0)mlflow.log_metric('metric',1.0)##### new APIs mlflow_extend provides ###### flatten dictmlflow.log_params_flatten({"a":{"b":0}})mlflow.log_metrics_flatten({"a":{"b":0.0}})# dictmlflow.log_dict({'a':0},'dict.json')# numpy arraymlflow.log_numpy(np.array([0]),'array.npy')# pandas dataframemlflow.log_df(pd.DataFrame({'a':[0]}),'df.csv')# matplotlib figurefig,ax=plt.subplots()ax.plot([0,1],[0,1])mlflow.log_figure(fig,'figure.png')# plotly figurefig=go.Figure(data=[go.Bar(x=[1,2,3],y=[1,3,2])])mlflow.log_figure(fig,'figure.html')# confusion matrixmlflow.log_confusion_matrix([[1,2],[3,4]])# run "mlflow ui" and see the result.
Package: mlflow-gcp-iap-plugin
MLFlow GCP IAP PluginThis project will allow for using an MLFlow service endpoint which is protected with IAP by setting an environment variable:MLFLOW_IAP_CLIENT_ID.Installationpipinstallmlflow-gcp-iap-pluginUseimportmflowimportosos.environ["MLFLOW_IAP_CLIENT_ID"]=<CLIENT_ID_VALUE># Use mlflow logging as desired...
Package: mlflow-jfrog-artifactory
Internal Jfrog Artifactory store plugin for MLflowThis repository provides a MLflow plugin that allows users to use aGeneric Artifactory repositoryas the artifact store for MLflow.Implementation overviewartifactory: this package includes theJFrogArtifactRepositoryclass that is used to read and write artifacts from Aliyun OSS storage.setup.pyfile defines entrypoints that tell MLflow to automatically associate theartifactoryURIs with theJFrogArtifactRepositoryimplementation when theartifactorylibrary is installed. The entrypoints are configured as follows:entry_points={
        "mlflow.artifact_repository": [
        entry_points={
            "mlflow.artifact_repository": "artifactory=mlflow_artifactory_plugin.store.artifact.jfrog_artifact_repository:JFrogArtifactRepository",  # noqa
        ]
    },RunningUsageInstall by pip on both your client and the server, and then use MLflow as normal. The JFrog Artifactory store support will be provided automatically.The plugin implements all of the MLflow artifact store APIs.
It expects Artifactory Storage access credentials in theMLFLOW_ARTIFACTORY_ENDPOINT_URL,MLFLOW_ARTIFACTORY_KEYandMLFLOW_ARTIFACTORY_REPOenvironment variables,
so you must set these variables on both your client application and your MLflow tracking server.
To use Artifactory as an artifact store, an artifactory URI of the formartifactory://<path>must be provided, as shown in the example below:importmlflowimportmlflow.pyfuncclassMod(mlflow.pyfunc.PythonModel):defpredict(self,ctx,inp):return7exp_name="myexp"mlflow.create_experiment(exp_name,artifact_location="artifactory://mlflow-test/")mlflow.set_experiment(exp_name)mlflow.pyfunc.log_model('model_test',python_model=Mod())Fix SSL errorThis plugin help to connect to external services (here Jrog artifactory).If you are facing this error, you proabably need to pass the location of the certifact.ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1123)Self-signed SSL certificates are obviously not taken into account implicitly and you need to  specify in REQUESTS_CA_BUNDLE. You can get around this behaviour by explicitly merging the environment settings into your session. For a Linux system you can set it like this before running the script:export REQUESTS_CA_BUNDLE=/path/to/certificat/file.pem
Package: mlflow-mlserver-docker
Mlflow Mlserver DockerThis is a utility package for building mlserver docker images based on MLflow models on a remote
tracking server.Currently, the S3 backend is the only officially supported backend, but any backend should work just
fine as long as you have all the required dependencies installed. If you're interested in other
backends let me know.Installationpip install mlflow-mlserver-dockerUsagemlflow-mlserver-dockerbuildruns:/fg8934ug54eg9hrdegu904/modelDevelopmentpoetryinstall
poetryselfaddpoetry-version-pluginMake suredockeris running.
Package: mlflow-observer
Observe yoursacredexperiments withmlflow.Writing experiments withsacredis great.mlflowprovides a nice UI that can be used to get a quick overview of your runs and analyze the results.UsageIn your code, add the observer:fromsacredimportExperimentfrommlflow_observerimportMlflowObserverfrom_pathsimportMY_TRACKING_URIex=Experiment('MyExperiment')ex.observers.append(MlflowObserver(MY_TRACKING_URI))In the commandline, you can pass a run name through sacred’s comment flag:pythontrain.py-c"My sacred run"Otherwise the run name will be of the formrun_[datetime].
Package: mlflow-oidc-client
MLflow Client OIDC/OAuth 2.1 PluginMLflow plugin adding OIDC/OAuth 2.1 authorization support to the client, allowing the use of a tracking server secured behind a compatible proxy.The plugin is built withOIDC Clientand supports the same OIDC/OAuth 2.1 authorization flows:theauthorization codeflow, for interactive user login;theclient credentialsflow, for confidential machine-to-machine communication.This plugin supports reading authorization settings frompyproject.toml.RequirementsPython 3.10+MLflow or MLflow Skinny 2+Installationpip install mlflow-oidc-clientGetting StartedFirst, add the following to the project'spyproject.tomlconfiguration file:[[tool.mlflow-oidc-client.tracking-servers]]uri="http://mlflow.example.com/"# URI of your MLflow Tracking Serverissuer="https://auth.example.com/"# URI of your OIDC providerclient-id="<application ID>"# Client ID of your projectYou can now run MLflow client commands without any change. The plugin will match theMLFLOW_TRACKING_URIenvironment variable to the appropriate server configuration found inpyproject.toml.#Tolistloggedexperiments:MLFLOW_TRACKING_URI=http://mlflow.example.com/ mlflow experiments searchConfigurationOptions may be set with environment variables or in thepyproject.tomlconfiguration file, with environment variables taking precedence.Each tracking server has its own[[tool.mlflow-oidc-client.tracking-servers]]block, which can be given multiple times in the samepyproject.toml.Environment VariableConfig FileDefault ValueDescriptionMLFLOW_TRACKING_URIN/AN/AMLflow Tracking Server URIMLFLOW_TRACKING_OIDC_ISSUERissuerNone(required)OIDC authorization issuer URIMLFLOW_TRACKING_OIDC_CLIENT_IDclient-idNone(required)OIDC client IDMLFLOW_TRACKING_OIDC_CLIENT_SECRETclient-secretNoneOIDC client secretMLFLOW_TRACKING_OIDC_REDIRECT_URIredirect-uri"http://127.0.0.1:39303/oauth2/callback"OIDC redirect URIMLFLOW_TRACKING_OIDC_SCOPEscope"openid profile email"OIDC token scopeMLFLOW_TRACKING_OIDC_AUDIENCEaudienceSame as the client IDOIDC token audienceMLFLOW_TRACKING_OIDC_INTERACTIVEinteractiveInteractive by default if the application is public (no client secret)Require a user login in a browserMLFLOW_TRACKING_OIDC_USE_ID_TOKENuse-id-tokenUse the ID token by default if the application is public (no client secret)Use the ID token instead of the access token asBearertoken in theAuthorizationHTTP headerExamplesBasic configuration providing interactive login for users:[[tool.mlflow-oidc-client.tracking-servers]]uri="http://mlflow.example.com/"issuer="https://auth.example.com/"client-id="<application ID>"Basic configuration for a machine-to-machine scenario (no interactive login required):[[tool.mlflow-oidc-client.tracking-servers]]uri="http://mlflow.example.com/"issuer="https://auth.example.com/"client-id="<application ID>"client-secret="<application ID>"audience="<audience>"# Required by some providers (e.g. Auth0)To avoid committing the client secret to git, you may pass it as theMLFLOW_TRACKING_OIDC_CLIENT_SECRETenvironment variable.LicenseThis project is licensed under the terms of the MIT license.AyzrFree and Open Source project.
Package: mlflowops
mlflowops UtilMLOps utils with MLFlowSetupQuick Installpython-mpipinstallmlflowopsBuild from sourceClone the repositorygitclonehttps://github.com/Broomva/mlflowops.gitInstall the packagecdmlflowops&&makeinstallBuild manuallyAfter cloning, create a virtual environmentcondacreate-nmlflowopspython=3.10
condaactivatemlflowopsInstall the requirementspipinstall-rrequirements.txtRun the python installationpythonsetup.pyinstallUsageThe deployment requires a .env file created under local folder:touch.envIt should have a schema like this:databricks_experiment_name=''databricks_experiment_id=''databricks_host=''databricks_token=''databricks_username=''databricks_password=''databricks_cluster_id=''importmlflowops
Package: mlflow-oss-artifact
Xkool Aliyun OSS store plugin for MLflowForked and modified from SeaOfOcean athttps://github.com/SeaOfOcean/mlflow-aliyunstoreThis repository provides a MLflow plugin that allows users to use Aliyun OSS as the artifact store for MLflow.UsagePip install the package on both your client and the serverpipinstallmlflow_oss_artifactConfigure environment variables in your OS for Aliyun OSS authenticationNote: checkoutthis poston stackoverflow to make them permanent if necessaryexportMLFLOW_OSS_ENDPOINT_URL=<oss-xx-cityname.aliyuncs.com>exportMLFLOW_OSS_KEY_ID=<your_oss_key_id>exportMLFLOW_OSS_KEY_SECRET=<your_oss_key_secret>exportMLFLOW_OSS_BUCKET_NAME=<your_bucket_name>To use To use Aliyun OSS as an artifact store, an OSS URI of the formoss://<path>must be provided, as shown in the example below:importmlflowimportmlflow.pyfuncclassMod(mlflow.pyfunc.PythonModel):defpredict(self,ctx,inp):return7exp_name="myexp"mlflow.create_experiment(exp_name,artifact_location="oss://mlflow-test/")mlflow.set_experiment(exp_name)mlflow.pyfunc.log_model('model_test',python_model=Mod())In the example provided above, thelog_modeloperation creates three entries in the OSS storageoss://mlflow-test/$RUN_ID/artifacts/model_test/, the MLmodel file
and the conda.yaml file associated with the model.
Package: mlflow-oss-store
MLflow artifact store plugin for AliCloud OSSYet another MLflow artifact store plugin for AliCloud OSS.Somehowhttps://github.com/mrly16/mlflow-aliyunstoreandhttps://github.com/SeaOfOcean/mlflow-aliyunstoredo not work for me. So I spent a rainy day to make another one.
Package: mlflow-plugin-proxy-auth
Mlflow Plugin Proxy AuthProvides authentication to Mlflow server usingProxy-Authorizationheader.UsageInstall the package into your virtual environment.pip install mlflow-plugin-proxy-authSet up mlflow environment variablesMLFLOW_PROXY_USERNAME=username
MLFLOW_PROXY_PASSWORD=password
MLFLOW_TRACKING_AUTH="proxy_auth_provider"
Package: mlflow-pyfunc-server
MLflow PyFunc ServerA simple server to publish several mlflow pyfunc models quickly in the private network.Settings can be defined in a config file (./mlflow-pyfunc-server.cfg, ~/.mlflow-pyfunc-server.cfg) or at startup:mlflow-pyfunc-server --host 0.0.0.0 -w 2 --mlflow https://mlflow:1234Develop$ py -m venv env
$ .\env\Scripts\activate
$ pip install -r requirements.txt