Tableau Server impact analysis reports: accessing metadata | by Elliott Stam | Towards Data Science

All good adventures begin with a first step. In this tutorial, the first step we take to build an impact analysis report for Tableau Server is collecting information about the relationships between our workbooks, visuals, and underlying database assets.
If you have no idea what I’m talking about, take a few minutes to check out the intro post, which kicks off this series of tutorials demonstrating how we can build impact analysis reporting for Tableau.
To summarize, our goal is to pull valuable metadata that exists in every Tableau ecosystem and provide an outline for how you can spin that data into insights. This is all about improving the productivity of our data teams and providing tailored visual information about our data lineage that anyone capable of clicking a mouse can easily interact with.
So let’s get started!
I want to get into the how as quickly as possible, but explaining the what is important. Our first step requires us to get our hands on some crucial information that will allow us to answer these types of questions:
How many of our visuals will be impacted if a specific database asset goes down? How can we know exactly what data feeds into our Tableau Server environment?Are the impacted visuals dashboards or just worksheets? What types of connections do they have, and are these connections embedded in individual workbooks or are they published?Do we have a lot of duplicated connections that could be consolidated into published datasources for better scalability and governance?Considering all the database connections we use, which specific tables are the most critical? How many Tableau workbooks reference those tables?
How many of our visuals will be impacted if a specific database asset goes down? How can we know exactly what data feeds into our Tableau Server environment?
Are the impacted visuals dashboards or just worksheets? What types of connections do they have, and are these connections embedded in individual workbooks or are they published?
Do we have a lot of duplicated connections that could be consolidated into published datasources for better scalability and governance?
Considering all the database connections we use, which specific tables are the most critical? How many Tableau workbooks reference those tables?
We could go on, but I think you get the point. The repeating themes of these questions are: workbooks, datasources, and database tables.
Let’s dive into this tutorial’s mission: pulling data from the Metadata API and shaping it into something that gets us closer to answering the questions above.
The saying goes, “there’s more than one way to skin a cat.” I’ve always liked cats, so I never loved this phrase. The reason I bring it up is that it’s wildly appropriate for the situation we find ourselves in with Tableau’s Metadata API.
For starters, you can take the no-code approach, or you can flip that on its head and do everything with code. Yep, there’s something for everyone.
We’ll begin with the no-code approach. You can use this to throw together a quick proof of concept or rely on it as your go-to data collection method if it provides exactly what you need and you don’t mind a bit of manual labor.
After covering the no-code approach, we’ll stretch our Python muscles and pave the way for you to integrate the Metadata API into automated workflows. Going the Python route will also allow us to do some nifty things such as enriching our metadata with contextual information available through other sources.
Regardless of your chosen method, you’re going to need to be on Tableau Server 2019.3 or later in order to use the Metadata API. If you use Tableau Online, you’re good to go. Did you know you can request a free Tableau Online sandbox site by joining their developer community? Now you know.
That covers the scope of this tutorial, though I think I’ll toss in one or two basic Tableau visuals at the end to prove that the data we pull is real and to highlight the value hiding within.
In future tutorials, Python will continue to emerge as a melting pot that will be our central focus for querying Metadata, the repository database, and the creation of .hyper extracts fueling our eventual impact analysis dashboards.
Without needing to write a script that authenticates into your Tableau Server environment and fetches information, you can go to this URL to access your metadata:
https://<your-tableau-server>.com/metadata/graphiql
You’ll be prompted to authenticate with your username and password as you normally would, but instead of seeing the usual interface you will land on the built-in GraphQL interface. By the way that is not a typo in the sample link above. It is ‘graphiql’ with an ‘i’ in it.
From here you can freestyle on building some basic GraphQL queries. Did I say no code earlier? Don’t worry, you’ll be fine. Feel free to just copy and paste these sample queries. If you’re the adventurous type, head over to the Metadata API documentation to master all the things.
For example, running this basic query will fetch you all of the workbook names, workbook IDs, and workbook owners on your site:
{  workbooks {    name    luid    owner {      name      email    }  }}
The response will appear on the right half of your GraphQL interface in JSON format. My output looked like this:
Did you know you can connect Tableau Desktop directly to JSON files? Yep, getting Metadata into a dashboard can be that easy. Just copy the JSON output and paste it into a text file. Save the file with a .json extension, and you’re set.
Below you’ll find the whole process in action, using a more advanced query in place of the toy example provided above. The query can be found in this Github Gist:
After you store the resulting JSON data to a .json file, open Tableau Desktop and connect to the file.
Select all the schemas available in the .json file and click ‘OK’.
That’s all it took — the data is in your hands now!
Doing things manually doesn’t cut it for everyone, myself included. Let’s cover how you can land at the same results we see above, but using code. This will set us up for nearly limitless customization as we look ahead to future tutorials, so I highly recommend investing time into the Python route.
I use the tableau-api-lib and pandas Python packages extensively in my Tableau Server automation workflows, so if you are following along then you will want to run this command in your command line:
pip install --upgrade tableau-api-libpip install --upgrade pandas
The full Python script relevant to this section can be found in this GitHub Gist:
New to Python? Download it and begin this life anew.
To briefly summarize what’s happening in that Python code, we are establishing a connection to a Tableau Server (or Tableau Online) site and running GraphQL queries against the Metadata API. The queries in this tutorial target workbooks, views, and their underlying database assets. You could of course expand upon these queries, and I encourage you to do so. We will certainly revisit this in later tutorials and build upon what’s covered here.
The GraphQL queries return JSON data. While we can feed that directly into Tableau, let’s plan ahead to future milestones in this project of ours where we want to combine our metadata with other sources of data, such as the repository database, which will provide all sorts of context including the last time someone interacted with our visuals.
To set ourselves up for future success, it’s worth taking the time to flatten our JSON data into multiple normalized tables and then combine those into a single ‘denormalized’ table. This will play very nicely with other one-off Metadata API queries and the various PostgreSQL queries we will put to work in upcoming tutorials.
Now, let’s toss in some visual context and see what this data can do for us.
To get a feel for the value this data can provide to us in terms of creating impact analysis reports, we already have a lot of the raw ingredients we need to cook up something useful.
With thirty seconds of drag-and-drop action, we’ve created the worksheet shown above! We can use the underlying data to build out customized data lineage audits, shame our least favorite business unit for using more worksheets than dashboards, and more.
We are now well on our way to being able to answer the questions laid out earlier in this tutorial. We’ve built a flashlight that can cast light on the darkest corners of our Tableau ecosystem.
That’s a wrap for our first milestone on this adventure building detailed and interactive impact analysis reports in Tableau. We’ve tapped into Tableau’s Metadata API and pulled enough data to build visuals describing the underlying database assets feeding our Tableau visuals. We demonstrated two methods to collect that data: manually via the built-in ‘metadata/graphiql’ interface, or with repeatable processes like Python scripts.
I hope this tutorial leaves you in a better place than when you started! Tune in for the next tutorial, where we’ll shift our focus to the repository database (the internal PostgreSQL database) and squeeze some really useful information out of it, including the number of times content was accessed by our users and the most recent interaction dates for each workbook or view.
The data we’ve collected is already valuable, but marrying it to the outputs from the PostgreSQL database will allow us to paint a more complete portrait of what’s happening in our Tableau ecosystem.
Which views and databases do our end-users access the most? Has any content gone stale and unused? Is it safe to drop that old database table John built four years ago? The work we’ll do in the next tutorial will help us answer those questions!
Milestone 2 (ETA is 4/29/2020): build upon Milestone 1 with supplemental data from the Tableau Server Repository to provide the number of interactions associated with each workbook (views and dashboards), datasource, and flow.
Milestone 3 (ETA is 5/6/2020): convert the combined output of Milestones 1 & 2 into a .hyper extract using the Hyper API and publish the contents to Tableau Server (or Tableau Online).
Milestone 4 (ETA is 5/13/2020): build our first impact analysis dashboard using the datasource published in Milestone 3.
Milestone 5 (TBD)