Package: temporary-cloud-governance
Cloud GovernanceWhat is it?Cloud Governancetool provides a lightweight and flexible framework for deploying cloud management policies focusing on cost optimize and security.This tool support the following policies:policyAWS PolicesReal time Openshift Cluster cost, User costec2_idle: idle ec2 in last 4 days, cpu < 2% & network < 5mb.ec2_run: running ec2.ebs_unattached: volumes that did not connect to instance, volume in available status.ebs_in_use: in use volumes.tag_resources: Update cluster and non cluster resource tags fetching from the user tags or from the mandatory tagszombie_cluster_resource: Delete cluster's zombie resourcestag_non_cluster: tag ec2 resources (instance, volume, ami, snapshot) by instance nametag_iam_user: update the user tags from the csv filecost_explorer: Get data from cost explorer and upload to ElasticSearchip_unattached: Get the unattached IP and delete it after 7 days.s3_inactive: Get the inactive/empty buckets and delete them after 7 days.empty_roles: Get empty roles and delete it after 7 days.zombie_snapshots: Get the zombie snapshots and delete it after 7 days.nat_gateway_unused: Get the unused nat gateways and deletes it after 7 days.gitleaks: scan Github repository git leak (security scan)cost_over_usage: send mail to aws user if over usage costIBM policiestag_baremetal: Tag IBM baremetal machinestag_vm: Tga IBM Virtual Machines machines** You can write your own policy usingCloud-Custodianand run it (see 'custom cloud custodian policy' inPolicy workflows).Reference:The cloud-governance package is placed inPyPiThe cloud-governance container image is placed inQuay.ioThe cloud-governance readthedocs link isReadTheDocsTable of ContentsInstallationConfigurationRun AWS Policy Using PodmanRun IBM Policy Using PodmanRun Policy Using PodPytestPost InstallationInstallationDownload cloud-governance image from quay.io# Need to run it with root privilegessudopodmanpullquay.io/ebattat/cloud-governanceEnvironment variables description:(mandatory)AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID(mandatory)AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEYPolicy name:(mandatory)policy=ec2_idle / ec2_run / ebs_unattached / ebs_in_use / tag_cluster_resource / zombie_cluster_resource / tag_ec2_resourcePolicy logs output(mandatory)policy_output=s3://redhat-cloud-governance/logsCluster or instance name:(mandatory policy:tag_cluster_resource)resource_name=ocs-testCluster or instance tags:(mandatory policy:tag_cluster_resource)mandatory_tags="{'Owner': 'Name','Email': 'name@redhat.com','Purpose': 'test'}"gitleaks(mandatory policy: gitleaks)git_access_token=$git_access_token
(mandatory policy: gitleaks)git_repo=https://github.com/redhat-performance/cloud-governance(optional policy: gitleaks)several_repos=yes/no (default = no)Choose a specific region or all for all the regions, default : us-east-2(optional)AWS_DEFAULT_REGION=us-east-2/all (default = us-east-2)Choose dry run or not, default yes(optional)dry_run=yes/no (default = yes)Choose log level, default INFO(optional)log_level=INFO (default = INFO)LDAP hostname to fetch mail recordsLDAP_HOST_NAME=ldap.example.comEnable Google Drive API in console and create Service accountGOOGLE_APPLICATION_CREDENTIALS=$pwd/service_account.jsonConfigurationAWS ConfigurationCreate a user and a bucketCreate user with IAMiamCreate a logs bucketcreate_bucket.shIBM ConfigurationCreate classic infrastructure API keyRun AWS Policy Using Podman# policy=ec2_idlesudopodmanrun--rm--namecloud-governance-epolicy="ec2_idle"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-edry_run="yes"-epolicy_output="s3://bucket/logs"-elog_level="INFO""quay.io/ebattat/cloud-governance"# policy=ec2_runsudopodmanrun--rm--namecloud-governance-epolicy="ec2_run"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-edry_run="yes"-epolicy_output="s3://bucket/logs"-elog_level="INFO""quay.io/ebattat/cloud-governance"# select policy ['ec2_stop', 's3_inactive', 'empty_roles', 'ip_unattached', 'nat_gateway_unused', 'zombie_snapshots']sudopodmanrun--rm--namecloud-governance-epolicy="policy"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-edry_run="yes"-elog_level="INFO""quay.io/ebattat/cloud-governance"# policy=ebs_unattachedsudopodmanrun--rm--namecloud-governance-epolicy="ebs_unattached"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-edry_run="yes"-epolicy_output="s3://bucket/logs"-elog_level="INFO""quay.io/ebattat/cloud-governance"# policy=ebs_in_usesudopodmanrun--rm--namecloud-governance-epolicy="ebs_in_use"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-edry_run="yes"-epolicy_output="s3://bucket/logs"-elog_level="INFO""quay.io/ebattat/cloud-governance"# policy=zombie_cluster_resourcesudopodmanrun--rm--namecloud-governance-epolicy="zombie_cluster_resource"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-edry_run="yes"-eresource="zombie_cluster_elastic_ip"-ecluster_tag="kubernetes.io/cluster/test-pd9qq"-elog_level="INFO""quay.io/ebattat/cloud-governance"# policy=tag_resourcessudopodmanrun--rm--namecloud-governance-epolicy="tag_resources"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-etag_operation="read/update/delete"-emandatory_tags="{'Owner': 'Name','Email': 'name@redhat.com','Purpose': 'test'}"-elog_level="INFO"-v"/etc/localtime":"/etc/localtime""quay.io/ebattat/cloud-governance"# policy=tag_non_clustersudopodmanrun--rm--namecloud-governance-epolicy="tag_non_cluster"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-etag_operation="read/update/delete"-emandatory_tags="{'Owner': 'Name','Email': 'name@redhat.com','Purpose': 'test'}"-elog_level="INFO"-v"/etc/localtime":"/etc/localtime""quay.io/ebattat/cloud-governance"# policy=tag_iam_usersudopodmanrun--rm--namecloud-governance-epolicy="tag_iam_user"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-euser_tag_operation="read/update/delete"-eremove_tags="['Environment', 'Test']"-eusername="test_username"-efile_name="tag_user.csv"-elog_level="INFO"-v"/home/user/tag_user.csv":"/tmp/tag_user.csv"--privileged"quay.io/ebattat/cloud-governance"# policy=cost_explorersudopodmanrun--rm--namecloud-governance-epolicy="cost_explorer"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-ees_host="$elasticsearch_host"-ees_port="$elasticsearch_port"-ees_index="$elasticsearch_index"-ecost_metric=UnblendedCost-estart_date="$start_date"-eend_date="$end_date"-egranularity="DAILY"-ecost_explorer_tags="['User', 'Budget', 'Project', 'Manager', 'Owner', 'LaunchTime', 'Name', 'Email']"-elog_level="INFO""quay.io/ebattat/cloud-governance:latest"sudopodmanrun--rm--namecloud-governance-epolicy="cost_explorer"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-ees_index="elasticsearch_index"-ecost_metric="UnblendedCost"-estart_date="$start_date"-eend_date="$end_date"-egranularity="DAILY"-ecost_explorer_tags="['User', 'Budget', 'Project', 'Manager', 'Owner', 'LaunchTime', 'Name', 'Email']"-efile_name="cost_explorer.txt"-v"/home/cost_explorer.txt":"/tmp/cost_explorer.txt"-elog_level="INFO""quay.io/ebattat/cloud-governance:latest"# policy=validate_iam_user_tagssudopodmanrun--rm--namecloud-governance-epolicy="validate_iam_user_tags"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-evalidate_type="spaces/tags"-euser_tags="['Budget', 'User', 'Owner', 'Manager', 'Environment', 'Project']"-elog_level="INFO""quay.io/ebattat/cloud-governance:latest"# policy=gitleakssudopodmanrun--rm--namecloud-governance-epolicy="gitleaks"-egit_access_token="$git_access_token"-egit_repo="https://github.com/redhat-performance/cloud-governance"-eseveral_repos="no"-elog_level="INFO""quay.io/ebattat/cloud-governance"# custom cloud custodian policy (path for custom policy: -v /home/user/custodian_policy:/custodian_policy)sudopodmanrun--rm--namecloud-governance-epolicy="/custodian_policy/policy.yml"-eAWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID"-eAWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY"-eAWS_DEFAULT_REGION="us-east-2"-edry_run="yes"-epolicy_output="s3://bucket/logs"-elog_level="INFO"-v"/home/user/custodian_policy":"/custodian_policy"--privileged"quay.io/ebattat/cloud-governance"Run IBM Policy Using Podman# policy=tag_baremetalpodmanrun--rm--namecloud-governance-epolicy="tag_baremetal"-eaccount="$account"-eIBM_API_USERNAME="$IBM_API_USERNAME"-eIBM_API_KEY="$IBM_API_KEY"-eSPREADSHEET_ID="$SPREADSHEET_ID"-eGOOGLE_APPLICATION_CREDENTIALS="$GOOGLE_APPLICATION_CREDENTIALS"-v$GOOGLE_APPLICATION_CREDENTIALS:$GOOGLE_APPLICATION_CREDENTIALS-eLDAP_USER_HOST="$LDAP_USER_HOST"-etag_operation="update"-elog_level="INFO"-v"/etc/localtime":"/etc/localtime""quay.io/ebattat/cloud-governance:latest"# tag=tab_vmpodmanrun--rm--namecloud-governance-epolicy="tag_vm"-eaccount="$account"-eIBM_API_USERNAME="$IBM_API_USERNAME"-eIBM_API_KEY="$IBM_API_KEY"-eSPREADSHEET_ID="$SPREADSHEET_ID"-eGOOGLE_APPLICATION_CREDENTIALS="$GOOGLE_APPLICATION_CREDENTIALS"-v$GOOGLE_APPLICATION_CREDENTIALS:$GOOGLE_APPLICATION_CREDENTIALS-eLDAP_USER_HOST="$LDAP_USER_HOST"-etag_operation="update"-elog_level="INFO"-v"/etc/localtime":"/etc/localtime""quay.io/ebattat/cloud-governance:latest"Run Policy Using PodRun as a pod job via OpenShiftJob Pod:cloud-governance.yamlConfigmaps:cloud_governance_configmap.yamlQuay.io Secret:quayio_secret.shAWS Secret:cloud_governance_secret.yaml* Need to convert secret key to base64 [run_base64.py](pod_yaml/run_base64.py)PytestCloud-governance integration tests using pytestpython3-mvenvgovernancesourcegovernance/bin/activate(governance)$python-mpipinstall--upgradepip(governance)$pipinstallcoverage(governance)$pipinstallpytest(governance)$gitclonehttps://github.com/redhat-performance/cloud-governance(governance)$cdcloud-governance(governance)$coveragerun-mpytest(governance)$deactivate
rm-rf*governance*Post InstallationDelete cloud-governance imagesudopodmanrmiquay.io/ebattat/cloud-governance