Accessing Azure DevOps APIs with large volumes of data
Tags: azuredevops, powershell
Published: 2021-12-14 18:51:59

---
title: Accessing Azure DevOps APIs with large volumes of data
published: true
date: 2021-09-27 14:29:46 UTC
tags: AzureDevOps,PowerShell
canonical_url: https://jessehouwing.net/azure-devops-accessing-apis-with-large-volumes-of-data/
---

> Note: When using PowerShell, be sure to escape the `$` in the query string, otherwise PowerShell will try to inject the value of whatever variable called `top` holds. You can escape the `$` with a preceding backtick:

```
$urlBase="...?api-version=6.0&`$top=5"
                              ^^^^^

```
 ![Accessing Azure DevOps APIs with large volumes of data](https://jessehouwing.net/content/images/2021/09/game-over-insert-coins.gif)

Using `$top=` has been tricky at times. It may or may not give you the requested number of items, it seems to depend on how busy the backend is. But it does promise you this: All the REST API's will return a `x-ms-continuation-token` header next to the payload and you can use this to fetch the next batch of items:

![Accessing Azure DevOps APIs with large volumes of data](https://i.stack.imgur.com/L9PNi.png)

You can fetch the next batch of projects by requesting the exact same REST query and adding the `&continuationToken=${ms-continuation-token}` query parameter to the call. Repeat this until the server stops sending the `x-ms-continuation-token` header, which signifies you've gotten all the values you were after.

You can grab the headers by passing in a second variable:

```
# Change top=5 below to 100 or so depending on your use case.
$urlBase="https://dev.azure.com/jessehouwing/_apis/projects?api-version=6.0&`$top=5"

$url = $urlBase
$results = @();

do 
{
    write-host "Calling API"
    $response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"} -Method Get -ContentType application/json -ResponseHeadersVariable headers
    $results += $response.value

    if ($headers["x-ms-continuationtoken"])
    {
        $continuation = $headers["x-ms-continuationtoken"]
        write-host "Token: $continuation"
        $url = $urlBase + "&continuationToken=" + $continuation
    }
} while ($headers["x-ms-continuationtoken"])

$results

```

When you run this, it will show:

```
Calling API
Token: 5
Calling API
Token: 10
Calling API

id : 06cb494c-b535-4f16-9323-bd0f63c38163
name : CMMI
url : https://dev.azure.com/jessehouwing/_apis/projects/06cb494c-b535-4f16-9323-bd0f63c38163
state : wellFormed
revision : 414360096
visibility : private
lastUpdateTime : 08/07/2019 20:19:09

id : 88a7ca79-b5c8-41d2-99e7-a5578a1df424
name : BattleJSip
description : Battleship case in JS/TS
url : https://dev.azure.com/jessehouwing/_apis/projects/88a7ca79-b5c8-41d2-99e7-a5578a1df424
state : wellFormed
revision : 414360059
visibility : private
lastUpdateTime : 15/04/2018 13:43:44

... for 12 projects

```

As you can see, 3 calls were made, and 2 continuation tokens were returned.

In case of the `/projects` endpoint the continuation token takes on a predictable value of the number of projects to skip. Other API's may return a GUID or a base64 encoded string or the ID of the last returned item. Make no assumptions about the contents of the token, always copy it from the header and put it into the next request verbatim.

Here's an example of what's passed over the wire:

```
GET https://dev.azure.com/p/_apis/projects?api-version=6.0&$top=5

x-ms-continuation-token:5

{"count":5,"value":[...]}
```

And then the next call will fetch projects 6 to 10 and sends a `continuation-token` back with the value 10 for the call after that:

```
GET https://dev.azure.com/p/_apis/projects?api-version=6.0&$top=5&continuationToken=5

x-ms-continuation-token:10

{"count":5,"value":[...]}
```

Until the `x-ms-continuation-token` header is no longer returned

```
GET https://dev.azure.com/p/_apis/projects?api-version=6.0&$top=5&continuationToken=10

{"count":2,"value":[...]}
```

[Originally answered on StackOverflow.](https://stackoverflow.com/questions/69345683/azure-devops-rest-api-top-parameter-in-powershell-script-is-not-working/69346262#69346262)