Chapter 2. Creating a filtered Microsoft Azure integration Note If you created an unfiltered Azure integration, do not complete the following steps. Your Azure integration is already complete. If you are using RHEL metering, after you integrate your data with cost management, go to Adding RHEL metering to a Microsoft Azure integration to finish configuring your integration for RHEL metering. Azure is a third-party product and its processes can change. The instructions for configuring third-party integrations are correct at the time of publishing. For the most up-to-date information, see Microsoft Azure's documentation . To share a subset of your billing data with Red Hat, you can configure a function script in Microsoft Azure. This script will filter your billing data and export it to object storage so that cost management can then access and read the filtered data. Add your Microsoft Azure integration to cost management from the Integrations page . To create an Azure integration, you will complete the following tasks: Create a storage account and resource group. Configure Storage Account Contributor and Reader roles for access. Create a function to filter the data that you want to send to Red Hat. 2.1. Selecting a scope and configuring roles In the Add a cloud integration wizard, you must select a scope to determine the level at which your cost data is collected and exported. If your scope requires the Billing account reader role, you must manually configure it in the Azure portal. The scope that you select determines where role-based access control permissions are applied. The most common Scope selection in the wizard is Subscription . The following list maps which role you need for each Scope selection: Cost Management Reader Azure RBAC role: Subscription Resource group Management group Billing account reader Azure RBAC role: Billing account Billing profile Invoice section Enrollment account If your scope requires the Cost Management reader role, you can run the commands as they are in the documentation. If your scope requires the Billing account reader role, see Assign Azure roles in the Azure portal to learn how to manually configure the role in the Azure portal. 2.2. Adding a Microsoft Azure account Add your Microsoft Azure account as an integration so that cost management can process the cost and usage data. Prerequisites You must have a Red Hat user account with Cloud Administrator entitlements. You must have a service account . Your service account must have the correct roles assigned in Hybrid Cloud Console to enable cost management access. For more information, see the User Access Configuration Guide . In cost management: Click Settings Menu > Integrations . In the Cloud tab, click Add integration . In the Add a cloud integration wizard, select Microsoft Azure and click . Enter a name for your integration and click . In the Select application step, select Cost management and click . In the Specify cost export scope step, select I wish to manually customize the data set sent to Cost Management . If you are registering RHEL usage billing, select Include RHEL usage . Otherwise, proceed to the step. Click . 2.3. Creating a Microsoft Azure resource group and storage account Create a storage account in Microsoft Azure to house your billing exports and a second storage account to house your filtered data. In your Microsoft Azure account : In the search bar, enter "storage" and click Storage accounts . On the Storage accounts page, click Create . In the Resource Group field, click Create new . Enter a name and click OK . In this example, use filtered-data-group . In the Instance details section, enter a name in the Storage account name field. For example, use filtereddata . Copy the names of the resource group and storage account so you can add them to Red Hat Hybrid Cloud Console later. Click Review . Review the storage account and click Create . In cost management: In the Add a cloud integration wizard, paste the resource group and storage account names that you copied into Resource group name and Storage account name . You will continue using the wizard in the following sections. 2.4. Creating a daily export in Microsoft Azure , set up an automatic export of your cost data to your Microsoft Azure storage account before you filter it for cost management. In your Microsoft Azure account : In the search bar, enter "cost exports" and click the result. Click Create . In Select a template , click Cost and usage (actual) to export your standard usage and purchase charges. Follow the steps in the Azure wizard: You can either create a new resource group and storage account or select existing ones. In this example, we use billingexportdata for the storage account and billinggroup for the resource group. You must set Format to CSV . Set Compression type to None or Gzip . Review the information and click Create . In cost management: Return to the Add a cloud integration wizard and complete the steps in Daily export Click . You will continue using the wizard in the following sections. 2.5. Finding your Microsoft Azure subscription ID Find your subscription_id in the Microsoft Azure Cloud Shell and add it to the Add a cloud integration wizard in cost management. In your Microsoft Azure account : Click Cloud Shell . Enter the following command to get your Subscription ID: az account show --query "{subscription_id: id }" Copy the value that is generated for subscription_id . Example response { "subscription_id": 00000000-0000-0000-000000000000 } In cost management: In the Subscription ID field of the Add a cloud integration wizard, paste the value that you copied in the step. Click . You will continue using the wizard in the following sections. 2.6. Creating Microsoft Azure roles for Red Hat access To grant Red Hat access to your data, you must configure dedicated roles in Microsoft Azure. If you have an additional resource under the same Azure subscription, you might not need to create a new service account. In cost management: In the Roles section of the Add a cloud integration wizard, copy the az ad sp create-for-rbac command to create a service principal with the Cost Management Storage Account Contributor role. In your Microsoft Azure account : Click Cloud Shell . In the cloud shell prompt, paste the command that you copied. Copy the values that are generated for the client ID, secret, and tenant: Example response { "client_id": "00000000-0000-0000-000000000000", "secret": "00000000-0000-0000-000000000000", "tenant": "00000000-0000-0000-000000000000" } In cost management: Return to the Add a cloud integration wizard and paste the values that you copied into their corresponding fields on the Roles page. Click . Review your information and click Add to complete your integration. In the pop-up screen that appears, copy the Source UUID for your function script. 2.7. Creating a function in Microsoft Azure Creating a function in Azure filters your data and adds it to the storage account that you created to share with Red Hat. You can use the example Python script in this section to gather and share the filtered cost data from your export. Prerequisites You must have Visual Studio Code installed on your device. You must have the Microsoft Azure functions extension installed in Visual Studio Code. To create an Azure function, Microsoft recommends that you use their Microsoft Visual Studio Code IDE to develop and deploy code. For more information about configuring Visual Studio Code, see Quickstart: Create a function in Azure with Python using Visual Studio Code . In your Microsoft Azure account : Enter functions in the search bar and select Function App . Click Create . Select a hosting option for your function and click Select . On the Create Function App page, add your resource group. In the Instance Details section, name your function app. In Runtime stack , select Python . In Version , select latest. Click Review + create : Click Create . Wait for the resource to be created and then click Go to resource to view. In Visual Studio Code: Click the Microsoft Azure tab and sign in to Azure. In the Workspaces drop-down, click Azure Functions which appears as an icon with an orange lightning bolt. Click Create Function . Follow the prompts to set a local location and select a language and version for your function. In this example, select Python , Model 2 and the latest version available. In Select a template for your function dialog, select Timer trigger , name the function, and then press enter. Set the cron expression to control when the function runs. In this example, use 0 9 * * * to run the function daily at 9 AM: Click Create . Click Open in the current window . In your requirements.txt file: After you create the function in your development environment, open the requirements.txt file, add the following requirements, and save the file: In init .py: Copy the Python script and paste it into` init .py`. Change the values in the section marked # Required vars to update to the values that correspond to your environment. The example script uses secrets from Azure Key Vaults to configure your service account client_id and client_secret as environment variables. You can alternatively enter your credentials directly into the script, although this is not best practice. The default script has built-in options for filtering your data or RHEL subscription filtering. You must uncomment the type of filtering you want to use or write your own custom filtering. Remove the comment from one of the following not both: filtered_data = hcs_filtering(df) filtered_data = rhel_filtering(df) If you want to write more customized filtering, you must include the following required columns: Some of the columns differ depending on the report type. The example script normalizes these columns and all filtered reports must follow this example. To filter the data, you must add dataframe filtering. For example: Exact matching: df.loc[(df["publishertype"] == "Marketplace")] Filters out all data that does not have a publisherType of Marketplace. Contains: df.loc[df["publishername"].astype(str).str.contains("Red Hat")] Filters all data that does not contain Red Hat in the publisherName . You can stack filters by using & (for AND) and | (for OR) with your df.loc clause. More useful filters: subscriptionid Filters specific subscriptions. resourcegroup Filters specific resource groups. resourcelocation Filters data in a specific region. You can use servicename , servicetier , metercategory and metersubcategory to filter specific service types. After you build your custom query, update the custom query in the example script under # custom filtering basic example # . Save the file. In Visual Studio Code: Right click the Function window and click Deploy to Function App . Select the function app that you created in the steps. 2.8. Setting up your credentials in Azure For help with steps in Azure, see Microsoft's documentation: Azure Key Vault . In your Microsoft Azure account : Navigate to Key Vaults . Click Create . Select the resource group that your function is in and follow the Azure wizard to create a new secret. In Key vault name , you can enter any name of your choosing. In Access policies , click Create new policy . Then, select Secret Management from the templates. In the tab Principal , search for and select your function as the principal. After you complete the wizard and click Create Vault , wait until Azure brings you to the successful deployment page. Then, click Go to resource to open the Key Vault page. In the Objects drop-down, select Secrets . You must create two new secrets for your service account: client_id and client_secret . Complete the following steps two times to create both: To create a secret, click Generate/import . In Create a secret , enter any name you want for your client_id or client_secret . Copy the value in Secret Identifier . You will use it later. You can also retrieve this value later. Repeat the three steps until you have a secret for both your client_id and client_secret . 2.9. Adding vault credentials to your function , go to your function in Microsoft Azure and enter information about your secrets. In your Microsoft Azure account : Navigate to your function. Select Settings Environment variables . Click Add . Use the following conventions and replace YOUR-CLIENT-ID-URI with the Secret Identifier value that you copied previously: Name: ClientIdFromVault Value: @Microsoft.KeyVault(SecretUri=YOUR-CLIENT-ID-URI) Click Save . Repeat the process for ClientSecretFromVault . Use the following conventions and replace YOUR-CLIENT-SECRET-URI with the Secret Identifier value that you copied previously: Value: @Microsoft.KeyVault(SecretUri=YOUR-CLIENT-SECRET-URI) 2.10. Configuring function roles in Microsoft Azure Configure dedicated credentials to grant your function blob access to Microsoft Azure cost data. These credentials enable your function to access, filter, and transfer the data from the original storage container to the filtered storage container. In your Microsoft Azure account : Enter functions in the search bar and select your function. In the Settings menu, click Identity . Complete the following set of steps twice , one time for each of the two storage accounts that you created in the section Creating a Microsoft Azure resource group and storage account: Click Azure role assignments . Click Add role assignment . In the Scope field, select Storage . In the Resource field, select one of your two storage accounts. Our examples used filtereddata and billingeportdata . In Role , select Storage Blob Data Contributor . Click Save . Click Add role assignment again. In the Scope field, select Storage . In the Resource field, select the same storage account again . This time, in Role , select Storage Queue Data Contributor . Click Save . Repeat this entire process for the other storage account that you created. After completing these steps, you have successfully set up your Azure integration. 2.11. Viewing your data You have now successfully created a filtered integration. To learn more about what you can do with your data, continue to steps for managing your costs .

Commands:
az account show --query "{subscription_id: id }"
{ "subscription_id": 00000000-0000-0000-000000000000 }
client_id: 00000000-0000-0000-000000000000
secret: 00000000-0000-0000-000000000000
tenant: 00000000-0000-0000-000000000000
azure-functions pandas requests azure-identity azure-storage-blob
'additionalinfo', 'billingaccountid', 'billingaccountname', 'billingcurrencycode', 'billingperiodenddate', 'billingperiodstartdate', 'chargetype', 'consumedservice', 'costinbillingcurrency', 'date', 'effectiveprice', 'metercategory', 'meterid', 'metername', 'meterregion', 'metersubcategory', 'offerid', 'productname', 'publishername', 'publishertype', 'quantity', 'reservationid', 'reservationname', 'resourcegroup', 'resourceid', 'resourcelocation', 'resourcename', 'servicefamily', 'serviceinfo1', 'serviceinfo2', 'subscriptionid', 'tags', 'unitofmeasure', 'unitprice'
column_translation = {"billingcurrency": "billingcurrencycode", "currency": "billingcurrencycode", "instanceid": "resourceid", "instancename": "resourceid", "pretaxcost": "costinbillingcurrency", "product": "productname", "resourcegroupname": "resourcegroup", "subscriptionguid": "subscriptionid", "servicename": "metercategory", "usage_quantity": "quantity"}