Package: azureml-infra-tools
azureml-infra-toolsazureml-infra-toolsis a Python utility package designed to simplify and streamline your Azure Machine Learning (Azure
ML) workflows. Our goal is to provide a set of high-level APIs that help set up Azure ML infrastructure, manage
authentication, and handle datasets in an intuitive and user-friendly manner.Whether you are an AI researcher, a data scientist, or a machine learning engineer,azureml-infra-toolscan help you
seamlessly leverage the power of Azure ML and accelerate your machine learning projects.You can check the project inGitHub page.
Also, you can find more information about Azure MLhere.InstallationYou can install theazureml-infra-toolspackage via pip:pipinstallazureml-infra-toolsOr via poetry:poetryaddazureml-infra-toolsUsageSimple usageHere is a simple example of how to use azureml-infra-tools:fromazureml_infra_toolsimportsetup_infrastructureimporthydrafromomegaconfimportDictConfig@hydra.main(version_base="1.2",config_path="conf",config_name="config")defmain(cfg:DictConfig):""" Main function to run the pipeline@param cfg: hydra configuration file"""# get original directory of the root of the projectoriginal_dir=hydra.utils.get_original_cwd()# create o setup with environment to run pipeline, client, cluster and dataazure_credential,data_asset,cpu_cluster,pipeline_env=setup_infrastructure(cfg,original_dir)Example of hydra configuration file:data:name:rpa-chatbot-assistant-intentions-csvdescription:RPA Chatbot Assistant Intentionsversion:0.1.0data_path:data/processed/data.csvcluster:name:rpachat-cluster-m60type:amlcomputesize:Standard_NV6min_instances:0max_instances:18idle_time_before_scale_down:180tier:Dedicatedenvironment:name:rpachat-custom-envdependencies_dir:src/dependenciesversion:1.2.0tags:{"datasets":"2.13.1","transformers":"4.30.2","torch":"2.0.1"}description:Custom environment for RPA Chatbot Assistant Intentions pipelineAdvanced usageHere is an advanced example of how to use azureml-infra-tools:# Get a handle to the workspaceazure_credential=AzureCredential(subscription_id=os.environ.get("SUBSCRIPTION_ID"),resource_group_name=os.environ.get("RESOURCE_GROUP_NAME"),workspace_name=os.environ.get("WORKSPACE_NAME"))# Upload the data to Azure ML Studiodata_asset:Data=AzureData(azure_credential=azure_credential,data_path=cfg.data.data_path,data_name=cfg.data.name,data_description=cfg.data.description,data_version=cfg.data.version).upload_data()# Create the clustercpu_cluster:AmlCompute=AzureCluster(azure_credential,cfg.cluster.name,cfg.cluster.type,cfg.cluster.size,cfg.cluster.min_instances,cfg.cluster.max_instances,cfg.cluster.idle_time_before_scale_down,cfg.cluster.tier).create_cluster()# Create the environmentpipeline_env:Environment=AzureEnvironment(azure_credential,cfg.environment.name,f"{original_dir}/{cfg.environment.dependencies_dir}",cfg.environment.version,cfg.environment.tags,cfg.environment.description).create_environment()NOTE: In both cases you need to create a .env file with SUBSCRIPTION_ID, RESOURCE_GROUP_NAME and WORKSPACE_NAME.
Then gather the information from the .env file and pass it to the AzureCredential class as follows:importosfromdotenvimportload_dotenvload_dotenv()os.environ.get("SUBSCRIPTION_ID")os.environ.get("RESOURCE_GROUP_NAME")os.environ.get("WORKSPACE_NAME")Please note that you need to provide your own directory and configuration parameters.ContributingWe appreciate all contributions. If you're planning to contribute back bug-fixes, please create an issue describing the
bug. If you plan to contribute new features, utility functions, or extensions, please first open an issue and discuss
the feature with us.LicenseThis project is licensed under the MIT License - see the LICENSE.md file for details.Please modify the contents to better match your project requirements and details. For instance, you might want to add
more usage examples, a section about the project's dependencies, instructions for how to run tests, etc.VersioningThe versioning system that we use is known as semantic versioning (SemVer). It's a versioning scheme for software
that aims to convey meaning about the underlying changes in a release.In general, SemVer's structure is MAJOR.MINOR.PATCH, where:MAJOR version increments indicate incompatible API changes.MINOR version increments indicate the addition of functionality in a backwards-compatible manner.PATCH version increments indicate backwards-compatible bug fixes.
For the versions available, see the tags on this repository.ContactFor more information on this project, you can visit the project'sGitHub page.