Finding storage usage of Confluence space and page using REST API
Asked by Jonny Klaas:

Hi,   

I try to fetch the consumed storage of our spaces by using the python script as documented here: <https://confluence.atlassian.com/confkb/finding-storage-usage-of-confluence-space-and-page-using-rest-api-1063555292.html>

The script is running fine but stops after 25 spaces. I can't find any limiter within the script, so maybe the REST API is limiting.

Can somebody help me out how to fetch all results?

Regards,

Jonny


Answers:
author: marc -Collabello--Phase Locked-
body: It seems the python script does not follow the \`_next\` links, if there are more than 25 results.

I believe the script needs to be modified for this.

comments: author: Patrick Alexander
body: Hi There,

is there anyone who has already updated the script?

Thanks

Patrick


author: Patrick Alexander
body: I got the Solution and edited the Script to get 500 results:

```
# This code sample uses the 'requests' 'json' 'csv' library:import requestsimport jsonimport csv#INSERT "USER", "TOKEN", "BASE_URL" HEREUSER="User"TOKEN="Token"BASE_URL="https://xxxxxxxxx.atlassian.net"with open('per_page.csv', 'w') as pagecsvfile, open('per_space.csv', 'w') as spacecsvfile:perPageWriter = csv.writer(pagecsvfile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)perSpaceWriter = csv.writer(spacecsvfile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)perPageWriter.writerow(['pageid','attachment_size(byte)'])perSpaceWriter.writerow(['space_name','space_key','attachment_size(byte)'])headers = {"Accept": "application/json"}response = requests.request("GET",BASE_URL + "/wiki/rest/api/space?limit=500",headers=headers,auth=(USER,TOKEN))site_attachment_volume = 0#Get all space keysspace_key_results = json.loads(response.text)["results"]for space in space_key_results:space_attachment_volume = 0#Get related page IDs from space keysresponse = requests.request("GET",BASE_URL + "/wiki/rest/api/space/" + space["key"] + "/content",headers=headers,auth=(USER,TOKEN))print("Space Key: " + space["key"])page_results = json.loads(response.text)["page"]["results"]for page in page_results:page_attachment_volume = 0#Get attachments from each pageprint(" " + "Page ID: " + page["id"])response = requests.request("GET",BASE_URL + "/wiki/rest/api/content/" + page["id"] + "/child/attachment",headers=headers,auth=(USER,TOKEN))attachment_results = json.loads(response.text)["results"]for attachment in attachment_results:print(" " + "Attachment Name: " + json.dumps(attachment["title"]) + ", " + json.dumps(attachment["extensions"]["fileSize"]) + " bytes")page_attachment_volume += int(json.dumps(attachment["extensions"]["fileSize"]))space_attachment_volume += page_attachment_volumeprint(" -->" + "PAGE TOTAL: " + str(page_attachment_volume))#Write to CSVperPageWriter.writerow([page["id"],str(page_attachment_volume)])print("\n " + "SPACE TOTAL: " + str(space_attachment_volume) + " bytes")print("----------")#Write to CSVperSpaceWriter.writerow([space["name"],space["key"],str(space_attachment_volume)])
```

comments: author: Jan
body: Hi there,

Would anyone be able to assist with updating this script to work with the v2 api from Atlassian for Confluence? Reason being I can use the pagination and response header to pull more than 250 requests which I need due to the amount of spaces.

I have updated the endpoints but am receiving the below error:

*** ** * ** ***

page_results = json.loads(response.text)\["page"\]\["results"\]  
\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^  
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads  
return _default_decoder.decode(s)  
\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^  
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode  
obj, end = self.raw_decode(s, idx=_w(s, 0).end())  
\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^  
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 355, in raw_decode  
raise JSONDecodeError("Expecting value", s, err.value) from None  
json.decoder.JSONDecodeError: Expecting value: line 3 column 1 (char 10)

*** ** * ** ***

Thanks

Jan


author: marc -Collabello--Phase Locked-
body: Hi,

Probably it is some programming work, as the v2 API is different from v1.

Do you need the output on the command line? Or is output in a table in Confluence also ok?


author: Jan
body: Hi,

Ideally if it could be in the same format where it saves to a CSV would be great, I only actually require the per_space.csv and not individual pages if that helps.

I am also not sure how the response header works in python as I have already have the command working for Get/Spaces working in Postman with the v2 api and see the response header at the bottom.

Thanks

Jan


Tags: cloud

confluence-cloud