Chapter 2. Creating a filtered Google Cloud integration Create a Google Cloud function script that can filter your billing data, store it in object storage, and send the filtered reports to cost management. Important If you created an unfiltered Azure integration, do not complete the following steps. Your Azure integration is already complete. You must have a Red Hat account user with Cloud Administrator permissions before you can add integrations to cost management. To create a Google Cloud integration, you will complete the following tasks: Create a Google Cloud project for your cost management data. Create a bucket for filtered reports. Create a billing service account member with the correct role to export your data to cost management. Create a BigQuery dataset that contains the cost data. Create a billing export that sends the cost management data to your BigQuery dataset. Note Google Cloud is a third-party product and its console and documentation can change. The instructions for configuring the third-party integrations are correct at the time of publishing. For the most up-to-date information, see the Google Cloud Platform documentation . 2.1. Adding your Google Cloud account as an integration You can add your Google Cloud account as an integration. After adding a Google Cloud integration, the cost management application processes the cost and usage data from your Google Cloud account and makes it viewable. Prerequisites To add data integrations to cost management, you must have a Red Hat account with Cloud Administrator permissions. Procedure From Red Hat Hybrid Cloud Console , click Settings Menu > Integrations . On the Settings page, in the Cloud tab, click Add integration . In the Add a cloud integration wizard, select Google Cloud as the cloud provider type and click . Enter a name for your integration. Click . In the Select application step, select Cost management and click . 2.2. Creating a Google Cloud project Create a Google Cloud project to gather and send your cost reports to Red Hat. Prerequisites Access to Google Cloud Console with resourcemanager.projects.create permission Procedure In the Google Cloud Console click IAM & Admin Create a Project . Enter a Project name in the new page that appears and select your billing account. Select the Organization . Enter the parent organization in the Location box. Click Create . In cost management: Pn the Project page, enter your Project ID . To configure Google Cloud to filter your data before it sends the data to Red Hat, select I wish to manually customize the data set sent to cost management . Click . Additional resources For additional information about creating projects, see the Google Cloud documentation Creating and managing projects . 2.3. Creating a Google Cloud bucket Create a bucket for filtered reports that you will create later. Buckets are containers that store data. In the Google Cloud Console : Go to Cloud Storage Buckets . Click Create . Enter your bucket information. Name your bucket. In this example, use customer-data . Click Create , then click Confirm in the confirmation dialog. In cost management: On the Create cloud storage bucket page, enter your Cloud storage bucket name . Additional resources For additional information about creating buckets, see the Google Cloud documentation on Creating buckets . 2.4. Creating a Google Cloud Identity and Access Management role A custom Identity and Access Management (IAM) role for cost management gives access to specific cost related resources required to enable a Google Cloud Platform integration and prohibits access to other resources. Prerequisites Access to Google Cloud Console with these permissions: resourcemanager.projects.get resourcemanager.projects.getIamPolicy resourcemanager.projects.setIamPolicy Google Cloud project Procedure In the Google Cloud Console , click IAM & Admin Roles . Select the project you created from the menu. Click + Create role . Enter a Title , Description and ID for the role. In this example, use customer-data-role . Click + ADD PERMISSIONS . Use the Enter property name or value field to search and select the following permissions for your custom role: storage.objects.get storage.objects.list storage.buckets.get Click ADD . Click CREATE . In the Add a cloud integration wizard, on the Create IAM role page, click . Additional resources For additional information about roles and their usage, see the Google Cloud documentation Understanding roles and Creating and managing custom roles . 2.5. Adding a billing service account member to your Google Cloud project You must create a billing service account member that can export cost reports to Red Hat Hybrid Cloud Console in your project. Prerequisites You must have access to Google Cloud Console and have the following permissions: resourcemanager.projects.get resourcemanager.projects.getIamPolicy resourcemanager.projects.setIamPolicy Google Cloud project A cost management Identity and Access Management (IAM) role In the Google Cloud Console : Click IAM & Admin IAM . Select the project you created from the menu. Click Grant Access . Paste the following principal into the New principals field: In the Assign roles section, assign the IAM role you created in Creating a Google Cloud Identity and Access Management role . In this example, use customer-data-role . Click SAVE . In the cost management: On the Assign access page, click . Verification steps Navigate to IAM & Admin IAM . Verify the new member is present with the correct role. Additional resources For additional information about roles and their usage, see the Google Cloud documentation Understanding roles and Creating and managing custom roles . 2.6. Creating a Google Cloud BigQuery dataset Create a BigQuery dataset to collect and store the billing data for cost management. Prerequisites Access to Google Cloud Console with bigquery.datasets.create permission Google Cloud project Procedure In Google Cloud Console , click BigQuery . In the Explorer panel, select the project you created. Click the action icon for your project name. Click CREATE DATASET . Enter a name for your dataset in the Dataset ID field. In this example, use CustomerFilteredData . Click CREATE DATASET . In the Add a cloud integration wizard, on the Create dataset page, enter the name of the dataset you created. Click . 2.7. Exporting Google Cloud billing data to BigQuery Enabling a billing export to BigQuery sends your Google Cloud billing data (such as usage, cost estimates, and pricing data) automatically to the BigQuery dataset you created in the last step. Prerequisites Access to Google Cloud Console with the Billing Account Administrator role Google Cloud project Billing service member with the cost management Identity and Access Management (IAM) role BigQuery dataset Procedure In the Google Cloud Console , click Billing Billing export . Click the Billing export tab. Click EDIT SETTINGS in the Detailed usage cost section. Select the cost management Project and Billing export dataset you created in the dropdown menus. Click SAVE . In the Add a cloud integration wizard, on the Billing export page, click . On the Review details page, review the information about your integration and click Add . Copy your source_uuid so that you can use it in the cloud function. Verification steps Verify a checkmark with Enabled in the Detailed usage cost section, with correct Project name and Dataset name . 2.8. Creating a function to post filtered data to your storage bucket Create a function that filters your data and adds it to the storage account that you created to share with Red Hat. You can use the example Python script to gather the cost data from your cost exports related to your Red Hat expenses and add it to the storage account. This script filters the cost data you created with BigQuery, removes non-Red Hat information, then creates .csv files, stores them in the bucket you created, and sends the data to Red Hat. Prerequisites You must have a Red Hat Hybrid Cloud Console service account . You must have enabled the API service in GCP. In the Google Cloud Console : Click Security Secret manager to set up a secret to authenticate your function with Red Hat without storing your credentials in your function. Enable the Secret Manager if it is not already enabled. From Secret Manager , click Create secret . Name your secret, add your service account Client ID, and click Create Secret . Repeat this process to save a secret for your service account Client secret. In the Google Cloud Console search bar, search for functions and select the Cloud Functions result. On the Cloud Functions page, click Create function . Name the function. In this example, use customer-data-function . In the Trigger section, select HTTPS as the trigger type. In Runtime, build, connections and security settings , click the Security and image repo tab. Click Add a secret reference . Select the client_id secret you created before. Set the reference method to Exposed as environment variable . Name the exposed environment variable client_id . Click Done . Repeat the steps for your client_secret . Click . On the Cloud Functions Code page, set the runtime to the latest Python version available. Open the requirements.txt file. Paste the following lines at the end of the file. Set the Entry Point to get_filtered_data . Open the main.py file. Paste the following python script . Change the values in the section marked # Required vars to update to the values for your environment. Update the values for the following lines: INTEGRATION_ID Cost management integration_id BUCKET Filtered data GCP Bucket PROJECT_ID Your project ID DATASET Your dataset name TABLE_ID Your table ID Click Deploy . 2.9. Trigger your function to post filtered data to your storage bucket Create a scheduler job to run the function you created to send filtered data to Red Hat on a schedule. Procedure Copy the Trigger URL for the function you created to post the cost reports. You will need to add it to the Google Cloud Scheduler. In the Google Cloud Console , search for functions and select the Cloud Functions result. On the Cloud Functions page, select your function, and click the Trigger tab. In the HTTP section, click Copy to clipboard . Create the scheduler job. In the Google Cloud Console , search for cloud scheduler and select the Cloud Scheduler result. Click Create job . Name your scheduler job. In this example, use CustomerFilteredDataSchedule . In the Frequency field, set the cron expression for when you want the function to run. In this example, use 0 9 * * * to run the function daily at 9 AM. Set the time zone and click Continue . Configure the execution on the page. In the Target type field, select HTTP . In the URL field, paste the Trigger URL you copied. In the body field, paste the following code that passes into the function to trigger it. {"name": "Scheduler"} In the Auth header field, select Add OIDC token . Click the Service account field and click Create to create a service account and role for the scheduler job. In the Service account details step, name your service account. In this example, use scheduler-service-account . Accept the default Service account ID and click Create and Continue . In the Grant this service account access to project field, search for and select Cloud Scheduler Job Runner as the first role. Click ADD ANOTHER ROLE , then search for and select Cloud Functions Invoker . Click Continue . Click Done to finish creating the service account. Go back to the Cloud scheduler tab. In the Configure the execution page, select the Service account field. Refresh the page and select the scheduler you just created. Click Continue and then click Create . After completing these steps, you have successfully set up your Google Cloud function to send reports to Red Hat. For steps, refer to Chapter 3, steps for managing your costs . 2.10. Creating additional cloud functions to collect finalized data At the beginning of the month, Google Cloud finalizes the bill for the month before. Create an additional function and scheduled job to trigger it to send these reports to Red Hat so cost management can process them. Procedure Set up a function to post reports: From Cloud Functions , select Create function . Name your function. Select HTTP trigger . In Runtime, build, connections, security settings , click Security . Click Reference secret . Select Exposed as environment variable . Select Secret version or Latest . Click Done . Repeat the process for your other secrets. Click Save . Copy your Trigger URL . Click . Select the latest Python runtime. Set Entry point to get_filtered_data . Add your Google Cloud function . Update the values for INTEGRATION_ID , BUCKET , PROJECT_ID , DATASET , and TABLE_ID . Remove the comments from the following lines: # month_end = now.replace(day=1) - timedelta(days=1) # delta = now.replace(day=1) - timedelta(days=query_range) # year = month_end.strftime("%Y") # month = month_end.strftime("%m") # day = month_end.strftime("%d") Select the requirements.py file and add the requirements from the requirements.txt file. Click Deploy . Set up a cloud scheduler to trigger your function: Go to Cloud Scheduler . Click Schedule a job . Name your schedule Set the frequency. For example, the following cron will run the job on the fourth day of every month, 0 9 4 * * Set a Time zone . Click Continue . Paste the function Trigger URL you copied earlier. In the request body, add {"name": "Scheduler"} . Set the auth header to OIDC token . Select or create a service account with the Cloud Scheudler Job Runner and Cloud Functions Invoker roles. Click Continue . Click Save .

Commands:
billing-export@red-hat-cost-management.iam.gserviceaccount.com
requests google-cloud-bigquery google-cloud-storage
name: Scheduler
month_end = now.replace(day=1) - timedelta(days=1) # delta = now.replace(day=1) - timedelta(days=query_range) # year = month_end.strftime("%Y") # month = month_end.strftime("%m") # day = month_end.strftime("%d")