Package: cloud-resource-matcher
Cloud Resource MatcherA framework to optimize cloud computing costs, usingmixed integer programming.This library should be used together withoptiframeandpulp.optiframeis the underlying optimization framework. This library provides modules for cloud computing that you can use withoptiframe.pulpis used to implement the mixed integer program. You only need to use it if you want to add additional modules or if you want to configure the solver.PrerequisitesPython>= 3.11optiframeInstallationpip install optiframeUsageThe ModulesThis library provides multiple modules that you can use inoptiframefor modeling cloud cost optimization problems:base_module: This module represents the basic decision of which cloud resources should be deployed on which cloud services.
It also adds instance demands and flat (upfront) base costs for cloud services.
This module must always be added, all other modules depend on it.performance_module: A module for performance requirements.
It allows you to define performance criteria (such as vCPUs and RAM) and the corresponding demand & supply.
Use-based pricing models can also be represented with this module.network_module: This module provides the means to encode network connections, maximum latency requirements and network traffic costs.multi_cloud_module: If multiple cloud service providers are considered for the decision, this module can be used.
It allows you to assign the cloud services to the providers, specify migration cost and enforce a minimum and maximum number of providers to be used.service_limits_module: If a cloud service is under very high demand and only a limited number of instances is available for purchase, this module can encode these requirements.Code ExampleHere is a small example demonstrating how to use this library:frompulpimportLpMinimizefromoptiframeimportOptimizer,SolutionObjValuefromcloud_resource_matcher.modules.baseimportBaseData,BaseSolution,base_modulefromcloud_resource_matcher.modules.performanceimportPerformanceData,performance_module# Specify the data of the problem instancebase_data=BaseData(...)performance_data=PerformanceData(...)solution=(Optimizer("cloud_cost_optimization",sense=LpMinimize)# Configure which modules you want to use.add_modules(base_module,performance_module)# Add the data of the problem instance.initialize(base_data,performance_data)# Obtain the optimal solution to the problem.solve())# Extract the total cost of the solutioncost=solution[SolutionObjValue].objective_value# Determine which cloud resource should be matched to which cloud servicematching=solution[BaseSolution]You can also take a look at theexamplesfolder for a more detailed example.
Thetest/case_studiesfolder also contains examples based on the pricing examples from cloud service providers.Configuring the SolverThis library usespulpunder the hood and is therefore agnostic to the solver backend that you can use.
By default, it uses the CBC solver, which is pre-bundled withpulp.
However, it's not very fast, so you probably want to change it.You can pass any solver object frompulpinto the.solve(...)method.
Take a look atthis documentationfor instructions on how to install and configure the solvers.GlossaryHere is a small glossary of terms that are used across this project:Cloud resource(CR): Anything you want to deploy on the cloud, such as virtual machines, serverless functions or databases.Cloud service(CS): An offer that you can buy in the cloud, for example Google Cloud C3, Amazon S3 or Azure Functions.Cloud service provider(CSP): A company offering cloud services. For example Google Cloud, AWS or Microsoft Azure.Mixed integer program(MIP): A mathematical description of optimization problems.
Solvers can use this to return an optimal solution to the problem.
See alsomixed integer programming.Module: A component that implements one set of decision criteria for the optimization problem.
For example, the network module implements functionality to represent network traffic and latencies.
By choosing the modules you want to use, you can configure the functionality of the optimizer.
If a decision criteria or pricing model you need to use is not implemented yet, you can define your own modules.Solver: A program implementing multiple algorithms to solve mixed integer programs to optimality.
Examples includeCBC(open source),SCIP(open source) andGurobi(commercial).DevelopmentWe usePoetryas a package manager, so you have to install it to properly run the project locally.
Then you can fork and clone the repository and runpoetry installto install all dependencies.We use several tools to ensure a consistent level of code quality:Runpoetry run pytestto run the test suit for the whole project.Runpoetry run mypy .to check for typing errors.Runpoetry run black .to format all Python files.Runpoetry run ruff .to lint all Python files.LicenseThis project is available under the terms of theMIT license