Power Automate - Flow Logging in App Insights
Tags: powerplatform, powerautomate, lowcode, rpa
Published: 2023-09-11 06:11:02

One of the big challenges with maintaining production flows in Power Automate was the lack of read only access. This meant to read a flow log you would need full edit access, not good for production environments using Service Accounts.

Fortunately Microsoft heard us and just launch [flow log integration with Application Insights](https://learn.microsoft.com/en-us/power-platform/admin/app-insights-cloud-flow). App Insights is the Azure standard for all logging, so it's incredibly powerful, additionally it can be setup with notification alerts and used as a data source for Power BI dashboards. It's not free (small memory cost and for alerts) and requires someone with Azure experience and permissions to create one.

Couple of call outs:

- It's not the same as the logs, as you do not see in flow data (the inputs and outputs, just the actions)
- The Global Power Platform Admin who creates the connection requires edit access to the app insight instance
- Each connection is between one environment and one app, but you can create multiple connections to each app, so you could have all your production flows linked to one App, all Dev another. Likewise you could link one environment to 2 apps, one for all Prod, one for the particular Dev,Test,Prod stack.
- In preview it works on any environment, once out of preview it will only work on Managed Environments (no surprise there lol)

## Setup
Setup is nice and easy, you need to be a Global Power Platform Admin, then you select following menus:
**Analytics - Data Export - App Insights - New Data Export**

![app insight config 1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7s9sjxxe9cr728auuky7.png)
Select Power Automate and runs, triggers, actions

![select environment](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0v45xhw85al4v6fosblv.png)
Select required environment


![select app insights](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oddufxop5koiqll5hbup.png)
Select subscription, resource group, from there you should see the list of available App Insights, select the required one

![app insight connections](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/aqi84jhzw8208zrnaab4.png)

You should then see list of all your connections, you can add more with the new data export from the top.

## App Insights
App Insights can be found in your Azure Portal

![app insights](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vvy64f5u7hydz1mxtf5u.png)

There is so much you can do with App Insights, like live data, alerts, and more. But I'm just going to look at the log query, as this is in my opinion the most useful.

![app insight query](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zaovm4iqaxjdtdnwmohz.png)

There are couple key commands that I use:

`let`- for variables `let myEnvironmentId = 'ca99eaab-99a9-ef99-9b99-99ded999999f';`

`requests` `dependencies` `traces` `exceptions` `customMetrics` - the table for what log you need, for Flows we just use requests and dependencies

` | where` - standard filter the results by `| where customDimensions ['environmentId'] == myEnvironmentId`

` | extend` - brings nested keys/fields to the root ` | extend Data = todynamic(tostring(customDimensions.Data))`

`| project` - selects required keys/fields for the report `| project timestamp ,id ,DisplayName = Data.FlowDisplayName`

## Request and Dependency Structure

There are a lot of keys/fields returned from the 2 tables, but to keep it simple I will identify the key ones:

**Requests**

| Key/Field            ﻿| Description                   | Node             |
|------------------|-------------------------------|------------------|
| timestamp        | date / time of run            |                  |
| id               | GUID for app insights         |                  |
| success          | boolean if run was successful |                  |
| name             | flow id                       |                  |
| duration         | time for flow to complete     |                  |
| itemType         | request or dependency         |                  |
| customDimensions | main node of data             |                  |
| Data             | node of data                  | customDimensions |
| FlowDisplayName  | flow name                     | Data             |
| RunId            | Run ID                        | Data             |
| tags             | node of data                  | Data             |
| createdBy        | flow owner id                 | tags             |
| environmentId    | environment id                | customDimensions |
| error            | node of data                  | customDimensions |
| code             | error description             | error            |
| message          | error reason                  | error            |
| signalCategory   | log type (cloud/desktop)      | customDimensions |

**Dependencies**

| Key/Field﻿            | Description                      | Node             |
|------------------|----------------------------------|------------------|
| timestamp        | date / time of run               |                  |
| id               | GUID for app insights            |                  |
| resourceId       | Flow ID                          | customDimensions|
| success          | boolean if action was successful |                  |
| name             | action name                      |                  |
| duration         | time for flow to complete        |                  |
| itemType         | request or dependency            |                  |
| RunId            | Run ID                           | Data             | 
| customDimensions | main node of data                |                  |
| Data             | node of data                     | customDimensions |
| FlowDisplayName  | flow name                        | Data             |
| operation_ParentId | Run ID                           | Data             |
| actionType       | action type                      | Data             |
| tags             | node of data                     | Data             |
| createdBy        | flow owner id                    | tags             |
| environmentId    | environment id                   | customDimensions |
| error            | node of data                     | customDimensions |
| code             | error description                | error            |
| message          | error reason                     | error            |
| signalCategory   | log type (cloud/desktop)         | customDimensions |

**Relationships**

![request to dependency relationships](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nefgbdx9okyqluscli45.png)


## Queries
You can see there are infinite different queries you can run, but I thought I would show the 3 that I have started to use:

**Flow Runs**
```
let myEnvironmentId = 'ca84eccb-78a7-ef84-9b20-84ded785680f'; 
requests 
| where customDimensions ['resourceProvider'] == 'Cloud Flow' 
| where customDimensions ['signalCategory'] == 'Cloud flow runs' 
| where customDimensions ['environmentId'] == myEnvironmentId 
| extend Data = todynamic(tostring(customDimensions.Data)) 
| extend Error = todynamic(tostring(customDimensions.error)) 
| project timestamp 
,id 
,DisplayName = Data.FlowDisplayName
,name
,RunID = Data.OriginRunId  
,ErrorCode = Error.code  
,ErrorMessage = Error.message  
,success 
,customDimensions
```

This query brings up all my flow runs (I use the time range in App Insights to narrow down time range). I also add `| where success== False` when I want just failed runs.

![flow runs results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nt9r1mfb7dg8cj46uzeq.png)

**Flow Action Performance**

```
let myEnvironmentId = 'ca99eaab-99a9-ef99-9b99-99ded999999f'; 
let myFlowId = '90911457-7dd5-453b-afb3-8f8f3374c599'; 
dependencies
| extend Data = todynamic(tostring(customDimensions.Data)) 
| extend DisplayName = Data.FlowDisplayName 
| extend RunID = Data.OriginRunId 
| extend Error = todynamic(tostring(customDimensions.error)) 
| extend ErrorCode = Error.code 
| extend ErrorMessage = Error.message 
| where name == 'Fail'
| where customDimensions.resourceId == myFlowId
| where customDimensions ['environmentId'] == myEnvironmentId 
| project timestamp 
,id 
,DisplayName 
,name
,operation_ParentId 
,ErrorCode
,ErrorMessage 
,success 
,customDimensions
```

This query allows me to look at how a specific action has performed over a set time span, helping debug potential issues.

![flow action results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vx841ca7i02wvmyj6ehl.png)

**Flow Detail**
```
let myEnvironmentId = 'ca99eaab-99a9-ef99-9b99-99ded999999f'; 
let myEnvironmentId = 'ca84eccb-78a7-ef84-9b20-84ded785680f'; 
let queryId='08585075362661487682209269813CU165'; 
(requests |union dependencies) 
| extend Data = todynamic(tostring(customDimensions.Data)) 
| extend DisplayName = Data.FlowDisplayName 
| extend RunID = Data.OriginRunId 
| extend Error = todynamic(tostring(customDimensions.error)) 
| extend ErrorCode = Error.code 
| extend ErrorMessage = Error.message 
| where operation_ParentId == queryId or RunID == queryId 
| where customDimensions ['environmentId'] == myEnvironmentId 
| project timestamp 
,id 
,DisplayName 
,name
,RunID 
,operation_ParentId 
,ErrorCode
,ErrorMessage 
,success 
,customDimensions
```

The above query allows me to view a specific flow run, with the request and dependencies union'd together.

![flow run results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u1rlgzcgxdn10e70p3uc.png)

## What's Next
So this is a good starting point to track flow performance and identify failed runs. But what is really cool is you can easily add the App Insights data to Power BI:

![app insights export](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ytqju7po2ggcb0sscmap.png)

Just click new dataset and it will open Power BI and create the connection and allow you to create reports with live data.

![power bi data connection](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/orid5at4qkwijq56kj51.png)
Below is auto generated, so you can do a lot more with some effort.
![power bi report](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j8cxyibo9otszdicq8vk.png)