How to Use Microsoft’s Graph API to Query, Create and Analyze Data from OneDrive, Outlook and more | by Dominik Polzer | Towards Data Science

From my emails, calendar entries, files in my cloud storage to the workgroups in Teams, a large part of my daily activities on my computer and phone are done with Microsoft applications. All these data are stored in the Microsoft Cloud, whether I like it or not. The good thing is that Microsoft provides an API that allows it to access and control most of this data, so why don`t use it to generate your own insights.
The following screenshot should give you an idea of what I want to convey with this article. I want to show you:
what my experiences are with the API itself and how you can build the queries you need to access the data
how the authentication process works, which gives your app the necessary permissions
how you can process and visualize the data in Django
Graph Quick Start — a fast way to start with Microsoft’s Graph API
OAuth 2.0 — How the login process works
Query Data using the Graph API
Analyze and visualize the data with Plotly
Summary
A few years ago, Microsoft switched from Live SDK to Microsoft Graph. Microsoft Graph is now the API to access all the data and insights in Microsoft 365. [1][2]
There are several ways you can start with Microsoft Graph API. The fastest way is to use the Graph Quick Start, which creates a new App ID and App Secret for the authentication process and already provides you with a first sample project.
(1) In the first step you need to define the language you want to use to build your App. Since I prefer Python, the example app will be a simple Django app.
If you are not familiar with Django, you can take a look at one of my previous articles, which explains the principle of Django as simply as possible and includes the most important points. In case you prefer to build your own app from scratch, you can also find a tutorial how to do that in the Microsoft docs.
towardsdatascience.com
(2) After selecting the language, the Quick Start gives you the option to directly create a new App ID with App Secret, which you will need later on for the authorization process.
To get an overview of your registered apps, you can do so in your Azure Portal under Azure Active Directory.
(3) The Quick Start provides you with a first sample app that already contains the basic structure and the authentication process. Download the code sample and save it in your project dictionary.
The structure of the code sample will look like any ordinary Django app. Furthermore, the YAML file oauth_settings.yml includes the settings for the authentication, such as required endpoints, App ID and App Secret.
(4) To be able to start the app for the first time, you have to install the necessary Python packages.
The requirements.txt provides an easy way to install a whole environment. [3] Since applications sometimes need specific versions of a library it makes sense to set up a Virtual Environment. [4] To do so, navigate into your project directory and execute the following command, which creates the Virtual Environment virtualenv_graph.
python -m venv virtualenv_graph
You will find a new directory virtualenv_graph in your project folder, which contains a copy of the Python interpreter. To actually work with the just created Environment graph-tutorial, you have to activate it by executing the batch file activate as follows:
graph-tutorial\Scripts\activate.bat
After activation, you will see the name of your virtual environment in brackets at the beginning of the input line of the CMD. This shows you that your environment is currently activated. With the following command, you can now install all necessary Python packages to your environment:
(5) The following command maps the model structure used in the app to the database:
python manage.py migrate
This will create some default tables for the used Django apps django.contrib.auth and django.contrib.session .
(6) The command python manage.py runserver starts a lightweight web server on our localhost. Unless otherwise specified, the local web server runs on port 8000 and the address 127.0.0.1.
You have now the possibility to just sign in with the login data of your Office365 account.
In the following I will briefly outline the functionality of OAuth 2.0 to explain what the given sample code does.
OAuth 2.0 is a industry-standard protocol for authentication and allows users to grant a third-party application access to the users resources, without giving the app your long-term credentials. [5][6]
This is possible, since the OAuth authorization layer separates the role of the client from the resource owner. To understand that a bit better, I will try to explain briefly which roles are part of the OAuth workflow and how they relate to each other. [5]
Resource Owner: The entity that grants access to the resource. In this case it’s you, the end-user which grants the app access to the data in the Microsoft Cloud related to your Microsoft 365 account.
Resource Server: Where the protected resources are located.
Client: The Client is the application that is requesting access to the resource, in this case the Django App we just downloaded.
Authorization Server: This server gives the end-user (the Resource Owner) the possibility to authenticate him/herself. After the proper authorization, the Authorization Server issues an access token to the App.
In the following I would like to show you how the Authorization Workflow is initiated with Python using the Template App.
As described before, the application first needs to send a request to the Authentication Server. Therefore we are using the Microsoft identity platform implementation of OAuth 2.0. [7]
Clicking the Sign-in button calls the sign_in function in tutorial.views.py , which in turn uses the get_sign_in_url function to form the request URL.
In the first step of the authorization flow, the client app directs the users to the /authorize endpoint. The request contains the [7]:
Application ID, which were assigned to your app during the registration
Scope, which defines the permissions the app requests from the user.
Redirect URI, where authentication responses can be sent and received by your app.
All the parameters you need during the authorization process are stored in the YAML file oauth_settings.yml .
(3) After the user has authenticated him/herself and thus granted the app access, the Authentification Server response with the Authorization Code and redirects the user to the Redirect URI you specified. (4) Together with your App ID and App Secret you can ask the Authorization Server for an Access Token and Refresh Token . (5) The access token then gives you the ability to access the API, at least as far as the required permissions were defined in scope .
That gives us access to the requested resources. In the following I will show you how to retrieve the data you are looking for.
To find the right data endpoints we can use Graph Explorer . It already gives you some requests about your account and various Microsoft applications. If you are looking for information that the given queries do not provide, it is worth taking a look at the docs. (e.g. for OneDrive)
I am specifically interested in data to my One Drive storage. However, the concept is the same for SharePoint, OneNote, Outlook, Teams and more.
You may need to add the proper permissions to the scope variable in oauth_settings.yml . You will find an overview of the permissions in the docs. For reading and writing files in One Drive add Files.ReadWrite.All to the scope . As an example, I would like to retrieve the information to all files in the folder /Bilder/Eigene Aufnahmen. Together with the just generated Access Token, I send a GET request to /drive/root:/<path>:/children to get the data to all files in the specified path.
The last line of the view function one_drive is rendering the HTML one_drive.html , which I now create to print the data I just got from the API. The HTML one_drive.html is similar to the calendar.html , which is already part of the sample app.
Now we just need to add the view function one_drive to the Router, the urls.py .
# <tutorial.urls.py>from django.urls import pathfrom . import viewsurlpatterns = [  ...  path('calendar', views.calendar, name='calendar'),  path('one_drive', views.one_drive, name='one_drive'),]
Now http://localhost:8000/one_drive shows me a HTML table with all files in the folder /Bilder/Eigene Aufnahme .
For the visualization of the data I use Plotly. If not already done so, install Plotly with pip as follows:
pip install plotly
In the following I am interested in the time when the pictures were taken. For the sake of clarity, I summarize the images by the calendar week in which they were taken.
Therefor I add a new key-values pair to the dictionary files['values'], which is defined as the calendar week of the creation date. Afterwards I transfer the values of the dictionary into a Pandas data frame. Pandas offers me a number of methods to manipulate the data frame, one of them I will use in the following. The operation groupby groups all entries based on the attribute CreationWeek, the method .count() counts the occurrences of the entries.
Visualized as a barplot, the whole thing looks like this. A simple example of how you can gradually build your own dashboard app, which shows you the most important key figures for your Microsoft services.
With this article I tried to share my experiences with the Microsoft Graph API:
the authentication process using the Microsoft identity platformquerying the data via the Graph APIvisualizing the data using Plotly
the authentication process using the Microsoft identity platform
querying the data via the Graph API
visualizing the data using Plotly
Beside retrieving data, the Graph interface offers a wide range of POST options. Add calendar entries, upload files or create a new Microsoft Team channel can be done via HTTP requests. This can help you to automate your processes.
[1] Overview of Microsoft Graph, https://docs.microsoft.com/en-us/graph/overview
[2] Migrating from Live SDK to Microsoft Graph, https://docs.microsoft.com/en-us/onedrive/developer/rest-api/concepts/migrating-from-live-sdk?view=odsp-graph-online
[3] Pip install, https://pip.pypa.io/en/stable/reference/pip_install/
[4] Virtual Environment and Packages, https://docs.python.org/3/tutorial/venv.html
[5] OAuth 2.0 Authorization Framework, https://auth0.com/docs/protocols/protocol-oauth2
[6] OAuth 2.0 Workflow, https://www.ibm.com/support/knowledgecenter/en/SS4J57_6.2.2.6/com.ibm.tivoli.fim.doc_6226/config/concept/OAuth20Workflow.html
[7] Microsoft identity platform and OAuth 2.0 authorization code flow, https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow
[8] Calling a secured API from a web server application, https://oauth.vlaanderen.be/authorization/Help/Api/AuthorizationCodeGrant