Repository: climatepolicyradar/navigator-data-ingest
Pull request #130: Doc to PDF conversion
Author: login: mpjuhasz
Created: 2025-03-19T13:07:10Z | Merged: 2025-03-24T14:44:53Z

## What this PR does
- adds libreoffice to the docker image
- adds a method that converts docx files to pdf in the upload stage, perstisting them as pdfs in the CDN
- minor refactor the `upload_document` method whilst I was there for clarity

see https://linear.app/climate-policy-radar/issue/SCI-217/parse-word-documents-520-docs

## How was this tested?
- running the conversion script locally on test documents and inspecting the resulting pdfs


Diff:
diff --git a/Dockerfile b/Dockerfile
index 6230ee66..6e998b59 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,5 +1,10 @@
 FROM python:3.10
 
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+    libreoffice \
+    && rm -rf /var/lib/apt/lists/*
+
 RUN mkdir /navigator-data-ingest
 WORKDIR /navigator-data-ingest
 
diff --git a/poetry.lock b/poetry.lock
index 81f23e58..26946b1d 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -1163,6 +1163,112 @@ files = [
     {file = "lazy_object_proxy-1.10.0-pp310.pp311.pp312.pp38.pp39-none-any.whl", hash = "sha256:80fa48bd89c8f2f456fc0765c11c23bf5af827febacd2f523ca5bc1893fcc09d"},
 ]
 
+[[package]]
+name = "levenshtein"
+version = "0.27.1"
+description = "Python extension for computing string edit distances and similarities."
+optional = false
+python-versions = ">=3.9"
+files = [
+    {file = "levenshtein-0.27.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:13d6f617cb6fe63714c4794861cfaacd398db58a292f930edb7f12aad931dace"},
+    {file = "levenshtein-0.27.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ca9d54d41075e130c390e61360bec80f116b62d6ae973aec502e77e921e95334"},
+    {file = "levenshtein-0.27.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2de1f822b5c9a20d10411f779dfd7181ce3407261436f8470008a98276a9d07f"},
+    {file = "levenshtein-0.27.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:81270392c2e45d1a7e1b3047c3a272d5e28bb4f1eff0137637980064948929b7"},
+    {file = "levenshtein-0.27.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2d30c3ea23a94dddd56dbe323e1fa8a29ceb24da18e2daa8d0abf78b269a5ad1"},
+    {file = "levenshtein-0.27.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0f3e0bea76695b9045bbf9ad5f67ad4cc01c11f783368f34760e068f19b6a6bc"},
+    {file = "levenshtein-0.27.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cdd190e468a68c31a5943368a5eaf4e130256a8707886d23ab5906a0cb98a43c"},
+    {file = "levenshtein-0.27.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:7c3121314bb4b676c011c33f6a0ebb462cfdcf378ff383e6f9e4cca5618d0ba7"},
+    {file = "levenshtein-0.27.1-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:f8ef378c873efcc5e978026b69b45342d841cd7a2f273447324f1c687cc4dc37"},
+    {file = "levenshtein-0.27.1-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:ff18d78c5c16bea20876425e1bf5af56c25918fb01bc0f2532db1317d4c0e157"},
+    {file = "levenshtein-0.27.1-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:13412ff805afbfe619d070280d1a76eb4198c60c5445cd5478bd4c7055bb3d51"},
+    {file = "levenshtein-0.27.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:a2adb9f263557f7fb13e19eb2f34595d86929a44c250b2fca6e9b65971e51e20"},
+    {file = "levenshtein-0.27.1-cp310-cp310-win32.whl", hash = "sha256:6278a33d2e0e909d8829b5a72191419c86dd3bb45b82399c7efc53dabe870c35"},
+    {file = "levenshtein-0.27.1-cp310-cp310-win_amd64.whl", hash = "sha256:5b602b8428ee5dc88432a55c5303a739ee2be7c15175bd67c29476a9d942f48e"},
+    {file = "levenshtein-0.27.1-cp310-cp310-win_arm64.whl", hash = "sha256:48334081fddaa0c259ba01ee898640a2cf8ede62e5f7e25fefece1c64d34837f"},
+    {file = "levenshtein-0.27.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:2e6f1760108319a108dceb2f02bc7cdb78807ad1f9c673c95eaa1d0fe5dfcaae"},
+    {file = "levenshtein-0.27.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:c4ed8400d94ab348099395e050b8ed9dd6a5d6b5b9e75e78b2b3d0b5f5b10f38"},
+    {file = "levenshtein-0.27.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7826efe51be8ff58bc44a633e022fdd4b9fc07396375a6dbc4945a3bffc7bf8f"},
+    {file = "levenshtein-0.27.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ff5afb78719659d353055863c7cb31599fbea6865c0890b2d840ee40214b3ddb"},
+    {file = "levenshtein-0.27.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:201dafd5c004cd52018560cf3213da799534d130cf0e4db839b51f3f06771de0"},
+    {file = "levenshtein-0.27.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e5ddd59f3cfaec216811ee67544779d9e2d6ed33f79337492a248245d6379e3d"},
+    {file = "levenshtein-0.27.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6afc241d27ecf5b921063b796812c55b0115423ca6fa4827aa4b1581643d0a65"},
+    {file = "levenshtein-0.27.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:ee2e766277cceb8ca9e584ea03b8dc064449ba588d3e24c1923e4b07576db574"},
+    {file = "levenshtein-0.27.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:920b23d6109453913ce78ec451bc402ff19d020ee8be4722e9d11192ec2fac6f"},
+    {file = "levenshtein-0.27.1-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:560d7edba126e2eea3ac3f2f12e7bd8bc9c6904089d12b5b23b6dfa98810b209"},
+    {file = "levenshtein-0.27.1-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:8d5362b6c7aa4896dc0cb1e7470a4ad3c06124e0af055dda30d81d3c5549346b"},
+    {file = "levenshtein-0.27.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:65ba880815b0f80a80a293aeebac0fab8069d03ad2d6f967a886063458f9d7a1"},
+    {file = "levenshtein-0.27.1-cp311-cp311-win32.whl", hash = "sha256:fcc08effe77fec0bc5b0f6f10ff20b9802b961c4a69047b5499f383119ddbe24"},
+    {file = "levenshtein-0.27.1-cp311-cp311-win_amd64.whl", hash = "sha256:0ed402d8902be7df212ac598fc189f9b2d520817fdbc6a05e2ce44f7f3ef6857"},
+    {file = "levenshtein-0.27.1-cp311-cp311-win_arm64.whl", hash = "sha256:7fdaab29af81a8eb981043737f42450efca64b9761ca29385487b29c506da5b5"},
+    {file = "levenshtein-0.27.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:25fb540d8c55d1dc7bdc59b7de518ea5ed9df92eb2077e74bcb9bb6de7b06f69"},
+    {file = "levenshtein-0.27.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f09cfab6387e9c908c7b37961c045e8e10eb9b7ec4a700367f8e080ee803a562"},
+    {file = "levenshtein-0.27.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:dafa29c0e616f322b574e0b2aeb5b1ff2f8d9a1a6550f22321f3bd9bb81036e3"},
+    {file = "levenshtein-0.27.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:be7a7642ea64392fa1e6ef7968c2e50ef2152c60948f95d0793361ed97cf8a6f"},
+    {file = "levenshtein-0.27.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:060b48c45ed54bcea9582ce79c6365b20a1a7473767e0b3d6be712fa3a22929c"},
+    {file = "levenshtein-0.27.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:712f562c5e64dd0398d3570fe99f8fbb88acec7cc431f101cb66c9d22d74c542"},
+    {file = "levenshtein-0.27.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a6141ad65cab49aa4527a3342d76c30c48adb2393b6cdfeca65caae8d25cb4b8"},
+    {file = "levenshtein-0.27.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:799b8d73cda3265331116f62932f553804eae16c706ceb35aaf16fc2a704791b"},
+    {file = "levenshtein-0.27.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:ec99871d98e517e1cc4a15659c62d6ea63ee5a2d72c5ddbebd7bae8b9e2670c8"},
+    {file = "levenshtein-0.27.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:8799164e1f83588dbdde07f728ea80796ea72196ea23484d78d891470241b222"},
+    {file = "levenshtein-0.27.1-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:583943813898326516ab451a83f734c6f07488cda5c361676150d3e3e8b47927"},
+    {file = "levenshtein-0.27.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5bb22956af44bb4eade93546bf95be610c8939b9a9d4d28b2dfa94abf454fed7"},
+    {file = "levenshtein-0.27.1-cp312-cp312-win32.whl", hash = "sha256:d9099ed1bcfa7ccc5540e8ad27b5dc6f23d16addcbe21fdd82af6440f4ed2b6d"},
+    {file = "levenshtein-0.27.1-cp312-cp312-win_amd64.whl", hash = "sha256:7f071ecdb50aa6c15fd8ae5bcb67e9da46ba1df7bba7c6bf6803a54c7a41fd96"},
+    {file = "levenshtein-0.27.1-cp312-cp312-win_arm64.whl", hash = "sha256:83b9033a984ccace7703f35b688f3907d55490182fd39b33a8e434d7b2e249e6"},
+    {file = "levenshtein-0.27.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ab00c2cae2889166afb7e1af64af2d4e8c1b126f3902d13ef3740df00e54032d"},
+    {file = "levenshtein-0.27.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c27e00bc7527e282f7c437817081df8da4eb7054e7ef9055b851fa3947896560"},
+    {file = "levenshtein-0.27.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a5b07de42bfc051136cc8e7f1e7ba2cb73666aa0429930f4218efabfdc5837ad"},
+    {file = "levenshtein-0.27.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:fb11ad3c9dae3063405aa50d9c96923722ab17bb606c776b6817d70b51fd7e07"},
+    {file = "levenshtein-0.27.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5c5986fb46cb0c063305fd45b0a79924abf2959a6d984bbac2b511d3ab259f3f"},
+    {file = "levenshtein-0.27.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:75191e469269ddef2859bc64c4a8cfd6c9e063302766b5cb7e1e67f38cc7051a"},
+    {file = "levenshtein-0.27.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:51b3a7b2266933babc04e4d9821a495142eebd6ef709f90e24bc532b52b81385"},
+    {file = "levenshtein-0.27.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:bbac509794afc3e2a9e73284c9e3d0aab5b1d928643f42b172969c3eefa1f2a3"},
+    {file = "levenshtein-0.27.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:8d68714785178347ecb272b94e85cbf7e638165895c4dd17ab57e7742d8872ec"},
+    {file = "levenshtein-0.27.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:8ee74ee31a5ab8f61cd6c6c6e9ade4488dde1285f3c12207afc018393c9b8d14"},
+    {file = "levenshtein-0.27.1-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:f2441b6365453ec89640b85344afd3d602b0d9972840b693508074c613486ce7"},
+    {file = "levenshtein-0.27.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:a9be39640a46d8a0f9be729e641651d16a62b2c07d3f4468c36e1cc66b0183b9"},
+    {file = "levenshtein-0.27.1-cp313-cp313-win32.whl", hash = "sha256:a520af67d976761eb6580e7c026a07eb8f74f910f17ce60e98d6e492a1f126c7"},
+    {file = "levenshtein-0.27.1-cp313-cp313-win_amd64.whl", hash = "sha256:7dd60aa49c2d8d23e0ef6452c8329029f5d092f386a177e3385d315cabb78f2a"},
+    {file = "levenshtein-0.27.1-cp313-cp313-win_arm64.whl", hash = "sha256:149cd4f0baf5884ac5df625b7b0d281721b15de00f447080e38f5188106e1167"},
+    {file = "levenshtein-0.27.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:0c9231ac7c705a689f12f4fc70286fa698b9c9f06091fcb0daddb245e9259cbe"},
+    {file = "levenshtein-0.27.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:cf9ba080b1a8659d35c11dcfffc7f8c001028c2a3a7b7e6832348cdd60c53329"},
+    {file = "levenshtein-0.27.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:164e3184385caca94ef7da49d373edd7fb52d4253bcc5bd5b780213dae307dfb"},
+    {file = "levenshtein-0.27.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e6024d67de6efbd32aaaafd964864c7fee0569b960556de326c3619d1eeb2ba4"},
+    {file = "levenshtein-0.27.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6fbb234b3b04e04f7b3a2f678e24fd873c86c543d541e9df3ac9ec1cc809e732"},
+    {file = "levenshtein-0.27.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ffdd9056c7afb29aea00b85acdb93a3524e43852b934ebb9126c901506d7a1ed"},
+    {file = "levenshtein-0.27.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a1a0918243a313f481f4ba6a61f35767c1230395a187caeecf0be87a7c8f0624"},
+    {file = "levenshtein-0.27.1-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:c57655b20690ffa5168df7f4b7c6207c4ca917b700fb1b142a49749eb1cf37bb"},
+    {file = "levenshtein-0.27.1-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:079cc78de05d3ded6cf1c5e2c3eadeb1232e12d49be7d5824d66c92b28c3555a"},
+    {file = "levenshtein-0.27.1-cp39-cp39-musllinux_1_2_ppc64le.whl", hash = "sha256:ac28c4ced134c0fe2941230ce4fd5c423aa66339e735321665fb9ae970f03a32"},
+    {file = "levenshtein-0.27.1-cp39-cp39-musllinux_1_2_s390x.whl", hash = "sha256:a2f7688355b22db27588f53c922b4583b8b627c83a8340191bbae1fbbc0f5f56"},
+    {file = "levenshtein-0.27.1-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:654e8f016cb64ad27263d3364c6536e7644205f20d94748c8b94c586e3362a23"},
+    {file = "levenshtein-0.27.1-cp39-cp39-win32.whl", hash = "sha256:145e6e8744643a3764fed9ab4ab9d3e2b8e5f05d2bcd0ad7df6f22f27a9fbcd4"},
+    {file = "levenshtein-0.27.1-cp39-cp39-win_amd64.whl", hash = "sha256:612f0c90201c318dd113e7e97bd677e6e3e27eb740f242b7ae1a83f13c892b7e"},
+    {file = "levenshtein-0.27.1-cp39-cp39-win_arm64.whl", hash = "sha256:cde09ec5b3cc84a6737113b47e45392b331c136a9e8a8ead8626f3eacae936f8"},
+    {file = "levenshtein-0.27.1-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:c92a222ab95b8d903eae6d5e7d51fe6c999be021b647715c18d04d0b0880f463"},
+    {file = "levenshtein-0.27.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:71afc36b4ee950fa1140aff22ffda9e5e23280285858e1303260dbb2eabf342d"},
+    {file = "levenshtein-0.27.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:58b1daeebfc148a571f09cfe18c16911ea1eaaa9e51065c5f7e7acbc4b866afa"},
+    {file = "levenshtein-0.27.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:105edcb14797d95c77f69bad23104314715a64cafbf4b0e79d354a33d7b54d8d"},
+    {file = "levenshtein-0.27.1-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d9c58fb1ef8bdc8773d705fbacf628e12c3bb63ee4d065dda18a76e86042444a"},
+    {file = "levenshtein-0.27.1-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:e52270591854af67217103955a36bd7436b57c801e3354e73ba44d689ed93697"},
+    {file = "levenshtein-0.27.1-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:909b7b6bce27a4ec90576c9a9bd9af5a41308dfecf364b410e80b58038277bbe"},
+    {file = "levenshtein-0.27.1-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:d193a7f97b8c6a350e36ec58e41a627c06fa4157c3ce4b2b11d90cfc3c2ebb8f"},
+    {file = "levenshtein-0.27.1-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:614be316e3c06118705fae1f717f9072d35108e5fd4e66a7dd0e80356135340b"},
+    {file = "levenshtein-0.27.1-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:31fc0a5bb070722bdabb6f7e14955a294a4a968c68202d294699817f21545d22"},
+    {file = "levenshtein-0.27.1-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:9415aa5257227af543be65768a80c7a75e266c3c818468ce6914812f88f9c3df"},
+    {file = "levenshtein-0.27.1-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:7987ef006a3cf56a4532bd4c90c2d3b7b4ca9ad3bf8ae1ee5713c4a3bdfda913"},
+    {file = "levenshtein-0.27.1-pp39-pypy39_pp73-macosx_10_15_x86_64.whl", hash = "sha256:e67750653459a8567b5bb10e56e7069b83428d42ff5f306be821ef033b92d1a8"},
+    {file = "levenshtein-0.27.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:93344c2c3812f21fdc46bd9e57171684fc53dd107dae2f648d65ea6225d5ceaf"},
+    {file = "levenshtein-0.27.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:da4baef7e7460691006dd2ca6b9e371aecf135130f72fddfe1620ae740b68d94"},
+    {file = "levenshtein-0.27.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8141c8e5bf2bd76ae214c348ba382045d7ed9d0e7ce060a36fc59c6af4b41d48"},
+    {file = "levenshtein-0.27.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:773aa120be48c71e25c08d92a2108786e6537a24081049664463715926c76b86"},
+    {file = "levenshtein-0.27.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:f12a99138fb09eb5606ab9de61dd234dd82a7babba8f227b5dce0e3ae3a9eaf4"},
+    {file = "levenshtein-0.27.1.tar.gz", hash = "sha256:3e18b73564cfc846eec94dd13fab6cb006b5d2e0cc56bad1fd7d5585881302e3"},
+]
+
+[package.dependencies]
+rapidfuzz = ">=3.9.0,<4.0.0"
+
 [[package]]
 name = "markdown-it-py"
 version = "3.0.0"
@@ -2049,6 +2155,23 @@ files = [
 [package.extras]
 windows-terminal = ["colorama (>=0.4.6)"]
 
+[[package]]
+name = "pymupdf"
+version = "1.25.4"
+description = "A high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents."
+optional = false
+python-versions = ">=3.9"
+files = [
+    {file = "pymupdf-1.25.4-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:b39ab5fc0cc793f2a93908cb4dc13182e61e9ace3476850ebf0e505f65355475"},
+    {file = "pymupdf-1.25.4-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:72ac2a460e15a4b75fb22d0bdeb9eb9b3bd6ab44d5288e56f85ff9e4241d0830"},
+    {file = "pymupdf-1.25.4-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bd16e264438b01bf6f6b8722deb49df20c0a9a7be2b4a8e29dcb7b5183d8b42a"},
+    {file = "pymupdf-1.25.4-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:72817aa5f5023632346eee4aad0c32018d7879cf9a72e1820054dc9d8c48150f"},
+    {file = "pymupdf-1.25.4-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a8c23242d209512437187dc728001fc7c65c2dd9d37b50071f34c5cd745194a3"},
+    {file = "pymupdf-1.25.4-cp39-abi3-win32.whl", hash = "sha256:d90e6087f9b153999fa9f2c4709488fb0355de0302ec502a76c4fc0c0a0c535e"},
+    {file = "pymupdf-1.25.4-cp39-abi3-win_amd64.whl", hash = "sha256:af8fa1f7920b3b27343b2511cb9fc44baf2bad8b948db0b0f9d5b2fa589cb305"},
+    {file = "pymupdf-1.25.4.tar.gz", hash = "sha256:5f189466b68901055a9ddc77dc1c91cba081a60964f0caa6ff5b9b87001a0194"},
+]
+
 [[package]]
 name = "pyparsing"
 version = "3.1.4"
@@ -2142,6 +2265,20 @@ files = [
 [package.extras]
 cli = ["click (>=5.0)"]
 
+[[package]]
+name = "python-levenshtein"
+version = "0.27.1"
+description = "Python extension for computing string edit distances and similarities."
+optional = false
+python-versions = ">=3.9"
+files = [
+    {file = "python_levenshtein-0.27.1-py3-none-any.whl", hash = "sha256:e1a4bc2a70284b2ebc4c505646142fecd0f831e49aa04ed972995895aec57396"},
+    {file = "python_levenshtein-0.27.1.tar.gz", hash = "sha256:3a5314a011016d373d309a68e875fd029caaa692ad3f32e78319299648045f11"},
+]
+
+[package.dependencies]
+Levenshtein = "0.27.1"
+
 [[package]]
 name = "python-slugify"
 version = "8.0.4"
@@ -3147,4 +3284,4 @@ type = ["pytest-mypy"]
 [metadata]
 lock-version = "2.0"
 python-versions = "^3.10"
-content-hash = "7580e1de1beba44da221c3d4c83cdb6e6c10640b7212f1ed345d3e5b2a1b4ee8"
+content-hash = "849883652f28a48d9a176ab06ef922c37efa2ae85f6f831ef485cc5ddb7de3a8"
diff --git a/pyproject.toml b/pyproject.toml
index 29fb414a..709a57c4 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -24,6 +24,8 @@ requests = "^2.28.1"
 tenacity = "^9.0.0"
 json-logging = "^1.3.0"
 cpr-sdk = "^1.16.0"
+pymupdf = "^1.25.4"
+python-levenshtein = "^0.27.1"
 
 [tool.poetry.group.dev-dependencies.dependencies]
 black = "^25.1.0"
diff --git a/src/navigator_data_ingest/base/api_client.py b/src/navigator_data_ingest/base/api_client.py
index 88661e60..dfc7dba3 100644
--- a/src/navigator_data_ingest/base/api_client.py
+++ b/src/navigator_data_ingest/base/api_client.py
@@ -11,10 +11,13 @@
 from tenacity.stop import stop_after_attempt
 from tenacity.wait import wait_random_exponential
 
+from navigator_data_ingest.base.doc_to_pdf_conversion import convert_doc_to_pdf
 from navigator_data_ingest.base.types import (
     FILE_EXTENSION_MAPPING,
     MULTI_FILE_CONTENT_TYPES,
     SUPPORTED_CONTENT_TYPES,
+    CONTENT_TYPE_DOCX,
+    CONTENT_TYPE_PDF,
     UnsupportedContentTypeError,
     UploadResult,
 )
@@ -47,6 +50,7 @@ def upload_document(
 
     :param requests.Session session: The session used for making the request.
     :return DocumentUploadResult: the remote URL and the md5_sum of its contents
+    :raises UnsupportedContentTypeError: if the content is of type multi-file or not within the list of supported
     """
     # download the document
     _LOGGER.info(f"Downloading document from '{source_url}' for {import_id}")
@@ -59,46 +63,28 @@ def upload_document(
     try:
         download_response = _download_from_source(session, source_url)
         content_type = determine_content_type(download_response, source_url)
+        file_content = download_response.content
 
-        # Update the result object with the detected content type
-        upload_result.content_type = content_type
+        # If the content type is DOCX, convert it to PDF
+        if content_type == CONTENT_TYPE_DOCX:
+            content_type = CONTENT_TYPE_PDF
+            file_content = convert_doc_to_pdf(file_content)
 
-        # Decide what to do next based on content type
-        if content_type in MULTI_FILE_CONTENT_TYPES:
-            raise UnsupportedContentTypeError(content_type)
+        upload_result.content_type = content_type
 
-        if content_type not in SUPPORTED_CONTENT_TYPES:
+        if (
+            content_type in MULTI_FILE_CONTENT_TYPES
+            or content_type not in SUPPORTED_CONTENT_TYPES
+        ):
             raise UnsupportedContentTypeError(content_type)
 
-        # Ensure valid file types can be read accordingly
-        file_content = download_response.content
-
         # Calculate the m5sum & update the result object with the calculated value
         file_hash = hashlib.md5(file_content).hexdigest()
         upload_result.md5_sum = file_hash
-
-        # ext4 used in Amazon Linux /tmp directory has a max filename length of
-        # 255 bytes, so trim to ensure we don't exceed that. Choose 240 initially to
-        # allow for suffix.
-        file_name_max_fs_len_no_suffix = file_name_without_suffix[:200]
-        while len(file_name_max_fs_len_no_suffix.encode("utf-8")) > 200:
-            file_name_max_fs_len_no_suffix = file_name_max_fs_len_no_suffix[:-5]
-
-        # s3 can only handle paths of up to 1024 bytes. To ensure we don't exceed that,
-        # we trim the filename if it's too long
         file_suffix = FILE_EXTENSION_MAPPING.get(content_type, "")
-        filename_max_len = (
-            1024
-            - len(s3_prefix)
-            - len(file_suffix)
-            - len(file_hash)
-            - len("_.")  # length of additional characters for joining path components
-        )
-        file_name_no_suffix_trimmed = file_name_max_fs_len_no_suffix[:filename_max_len]
-        # Safe not to loop over the encoding of file_name because everything we're
-        # adding is 1byte == 1char
-        file_name = (
-            f"{s3_prefix}/{file_name_no_suffix_trimmed}_{file_hash}{file_suffix}"
+
+        file_name = _create_file_name_for_upload(
+            file_hash, file_name_without_suffix, file_suffix, s3_prefix
         )
 
         _LOGGER.info(
@@ -124,6 +110,34 @@ def upload_document(
         return upload_result
 
 
+def _create_file_name_for_upload(
+    file_hash: str, file_name_without_suffix: str, file_suffix: str, s3_prefix: str
+) -> str:
+    """Constructs a trimmed and enriched file name for uploading to S3"""
+    # ext4 used in Amazon Linux /tmp directory has a max filename length of
+    # 255 bytes, so trim to ensure we don't exceed that. Choose 240 initially to
+    # allow for suffix.
+    file_name_max_fs_len_no_suffix = file_name_without_suffix[:200]
+    while len(file_name_max_fs_len_no_suffix.encode("utf-8")) > 200:
+        file_name_max_fs_len_no_suffix = file_name_max_fs_len_no_suffix[:-5]
+
+    # s3 can only handle paths of up to 1024 bytes. To ensure we don't exceed that,
+    # we trim the filename if it's too long
+    file_name_max_len = (
+        1024
+        - len(s3_prefix)
+        - len(file_suffix)
+        - len(file_hash)
+        - len("_.")  # length of additional characters for joining path components
+    )
+    file_name_no_suffix_trimmed = file_name_max_fs_len_no_suffix[:file_name_max_len]
+    # Safe not to loop over the encoding of file_name because everything we're
+    # adding is 1byte == 1char
+    file_name = f"{s3_prefix}/{file_name_no_suffix_trimmed}_{file_hash}{file_suffix}"
+
+    return file_name
+
+
 @retry(
     stop=stop_after_attempt(4),
     wait=wait_random_exponential(multiplier=1, min=1, max=10),
diff --git a/src/navigator_data_ingest/base/doc_to_pdf_conversion.py b/src/navigator_data_ingest/base/doc_to_pdf_conversion.py
new file mode 100644
index 00000000..bde3e45b
--- /dev/null
+++ b/src/navigator_data_ingest/base/doc_to_pdf_conversion.py
@@ -0,0 +1,46 @@
+from pathlib import Path
+import shutil
+from tempfile import mkdtemp
+from uuid import uuid4
+
+import subprocess
+
+
+def convert_doc_to_pdf(file_content: bytes) -> bytes:
+    """
+    Transforms a docx / doc file into pdf, and returns the bytes file content
+
+    It uses a temporary directory to store the docx and pdf files.
+
+    :param file_content: the loaded file content of the docx file
+    :return: the file content of the pdf file
+    """
+    suffix = str(uuid4())
+    # to make this thread safe, create a unique directory for each worker
+    worker_dir = mkdtemp(prefix="worker_", suffix=suffix)
+
+    input_file_path = f"{worker_dir}/doc.docx"
+    output_file_path = f"{worker_dir}/doc.pdf"
+
+    with open(input_file_path, "wb") as input_file:
+        input_file.write(file_content)
+
+    cmd = [
+        "soffice",
+        "--headless",
+        "--convert-to",
+        "pdf",
+        "--outdir",
+        str(Path(output_file_path).parent),
+        input_file_path,
+    ]
+
+    process = subprocess.run(cmd, capture_output=True, text=True)
+    if process.returncode != 0:
+        raise RuntimeError(f"Conversion failed: {process.stderr}")
+
+    with open(output_file_path, "rb") as output_file:
+        pdf_file_content = output_file.read()
+
+    shutil.rmtree(worker_dir)
+    return pdf_file_content
diff --git a/src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.doc b/src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.doc
new file mode 100644
index 00000000..baab297c
Binary files /dev/null and b/src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.doc differ
diff --git a/src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.pdf b/src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.pdf
new file mode 100644
index 00000000..410483ab
Binary files /dev/null and b/src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.pdf differ
diff --git a/src/navigator_data_ingest/tests/test_api_client.py b/src/navigator_data_ingest/tests/test_api_client.py
index c524e772..dba70519 100644
--- a/src/navigator_data_ingest/tests/test_api_client.py
+++ b/src/navigator_data_ingest/tests/test_api_client.py
@@ -18,14 +18,14 @@ def test_upload_document__readable(
     requests_mock.get(url, content=pdf_bytes, headers={"content-type": content_type})
 
     result = upload_document(
-        session = session,
+        session=session,
         source_url=url,
         s3_prefix="TEST/1970",
         file_name_without_suffix="test_slug",
         document_bucket=mock_cdn_config["bucket"],
         import_id="TEST.0.1",
     )
-    
+
     assert result.content_type == content_type
     assert result.cdn_object.startswith("TEST/1970/test_slug")
     assert result.cdn_object.endswith(".pdf")
@@ -36,7 +36,7 @@ def test_upload_document__readable(
     ("url", "content_type", "data"),
     [
         ("mock://somedata", "text/html", b"<html></html>"),
-    ]
+    ],
 )
 def test_upload_document__unreadable(
     test_s3_client__cdn,
@@ -50,14 +50,14 @@ def test_upload_document__unreadable(
     requests_mock.get(url, content=data, headers={"content-type": content_type})
 
     result = upload_document(
-        session = session,
+        session=session,
         source_url=url,
         s3_prefix="TEST/1970",
         file_name_without_suffix="test_slug",
         document_bucket=mock_cdn_config["bucket"],
         import_id="TEST.0.1",
     )
-    
+
     assert result.md5_sum is None
     assert result.cdn_object is None
     assert result.content_type is None
diff --git a/src/navigator_data_ingest/tests/test_doc_to_pdf_conversion.py b/src/navigator_data_ingest/tests/test_doc_to_pdf_conversion.py
new file mode 100644
index 00000000..fb471e9d
--- /dev/null
+++ b/src/navigator_data_ingest/tests/test_doc_to_pdf_conversion.py
@@ -0,0 +1,42 @@
+import re
+import fitz
+
+from Levenshtein import distance
+
+from navigator_data_ingest.base.doc_to_pdf_conversion import convert_doc_to_pdf
+
+
+def all_text(doc: fitz.Document) -> str:
+    text = ""
+    for page in doc:
+        text += re.sub("( |\n|\t)+", " ", page.get_text().replace("\n", " "))
+    return text
+
+
+def test_convert_doc_to_pdf():
+    with open(
+        "src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.doc",
+        "rb",
+    ) as file:
+        doc_content = file.read()
+
+    pdf_content = convert_doc_to_pdf(doc_content)
+
+    converted_pdf = fitz.open(stream=pdf_content, filetype="pdf")
+    expected_pdf = fitz.open(
+        "src/navigator_data_ingest/tests/fixtures/sample-for-word-to-pdf-conversion.pdf"
+    )
+
+    # libreoffice messes with the pages -- the converted pdf is actually longer (67 pages vs 54)
+    # assert converted_pdf.page_count == expected_pdf.page_count
+    # this also results in content change: e.g. the page numbers are not the same. Tables in particular are dragged out
+    # into multiple pages on the conversion
+
+    assert (
+        distance(all_text(converted_pdf), all_text(expected_pdf))
+        / len(all_text(expected_pdf))
+        < 0.06
+    )
+    assert (len(all_text(converted_pdf)) - len(all_text(expected_pdf))) / len(
+        all_text(expected_pdf)
+    ) < 0.01
diff --git a/src/navigator_data_ingest/tests/test_utils.py b/src/navigator_data_ingest/tests/test_utils.py
index 85120e2f..c9ccfeeb 100644
--- a/src/navigator_data_ingest/tests/test_utils.py
+++ b/src/navigator_data_ingest/tests/test_utils.py
@@ -15,7 +15,7 @@
         ["application/pdf", "https://aweb.site/file.pdf", CONTENT_TYPE_PDF],
         ["", "https://aweb.site/file.pdf", CONTENT_TYPE_PDF],
         ["", "https://aweb.site/file", ""],
-    )
+    ),
 )
 def test_determine_content_type(content_type, source_url, want):
     test_response = Response()


Reviews:
user: login: kdutia
body: code looks good, and nice refactor! do we want a test for this new functionality too?
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-19T13:27:30Z
user: login: kdutia
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-20T15:48:31Z
user: login: kdutia
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-20T15:49:39Z
user: login: kdutia
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-20T15:50:55Z
user: login: kdutia
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-20T15:52:02Z
user: login: kdutia
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-24T14:44:52Z

Review comments:
diff_hunk: @@ -0,0 +1,43 @@
+import re
+import pytest
+import fitz
+
+from Levenshtein import distance
+
+from navigator_data_ingest.base.doc_to_pdf_conversion import convert_doc_to_pdf
+
+
+def all_text(doc: fitz.Document) -> str:
+    text = ""
+    for page in doc:
+        text += re.sub("( |\n|\t)+", " ", page.get_text().replace("\n", " "))
+    return text
+
+
+@pytest.mark.skip(reason="This test requires libreoffice in the environment")
path: src/navigator_data_ingest/tests/test_doc_to_pdf_conversion.py
user: login: kdutia
body: I don't think this test should be skipped, as we want it to run in CI?
created_at: 2025-03-20T15:48:31Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,43 @@
+import re
+import pytest
+import fitz
+
+from Levenshtein import distance
+
+from navigator_data_ingest.base.doc_to_pdf_conversion import convert_doc_to_pdf
+
+
+def all_text(doc: fitz.Document) -> str:
+    text = ""
+    for page in doc:
+        text += re.sub("( |\n|\t)+", " ", page.get_text().replace("\n", " "))
+    return text
+
+
+@pytest.mark.skip(reason="This test requires libreoffice in the environment")
+def test_convert_doc_to_pdf():
+    with open(
+        "src/navigator_data_ingest/tests/fixtures/sampe-01-for-conversion.doc", "rb"
path: src/navigator_data_ingest/tests/test_doc_to_pdf_conversion.py
user: login: kdutia
body: typo (also below, and nn in the filenames). Also, we probably want a more specific filename or a subdir that indicates these files are used for word to pdf testing?

```suggestion
        "src/navigator_data_ingest/tests/fixtures/sample-01-for-conversion.doc", "rb"
```
created_at: 2025-03-20T15:49:39Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,43 @@
+import re
+import pytest
+import fitz
+
+from Levenshtein import distance
+
+from navigator_data_ingest.base.doc_to_pdf_conversion import convert_doc_to_pdf
+
+
+def all_text(doc: fitz.Document) -> str:
+    text = ""
+    for page in doc:
+        text += re.sub("( |\n|\t)+", " ", page.get_text().replace("\n", " "))
+    return text
+
+
+@pytest.mark.skip(reason="This test requires libreoffice in the environment")
+def test_convert_doc_to_pdf():
+    with open(
+        "src/navigator_data_ingest/tests/fixtures/sampe-01-for-conversion.doc", "rb"
+    ) as file:
+        doc_content = file.read()
+
+    pdf_content = convert_doc_to_pdf(doc_content)
+
+    converted_pdf = fitz.open(stream=pdf_content, filetype="pdf")
+    expected_pdf = fitz.open(
+        "src/navigator_data_ingest/tests/fixtures/sampe-01-for-conversion.pdf"
+    )
+
+    # libreoffice messes with the pages -- the converted pdf is actually longer (67 pages vs 54)
+    # assert converted_pdf.page_count == expected_pdf.page_count
+    # this also results in content change: e.g. the page numbers are not the same. Tables in particular are dragged out
+    # into multiple pages on the conversion
+
+    assert (
+        distance(all_text(converted_pdf), all_text(expected_pdf))
+        / len(all_text(expected_pdf))
+        < 0.06
+    )
+    assert (len(all_text(converted_pdf)) - len(all_text(expected_pdf))) / len(
+        all_text(expected_pdf)
+    ) < 0.01
path: src/navigator_data_ingest/tests/test_doc_to_pdf_conversion.py
user: login: kdutia
body: nice!
created_at: 2025-03-20T15:50:55Z
author_association: MEMBER
diff_hunk: @@ -18,14 +18,14 @@ def test_upload_document__readable(
     requests_mock.get(url, content=pdf_bytes, headers={"content-type": content_type})
 
     result = upload_document(
-        session = session,
+        session=session,
path: src/navigator_data_ingest/tests/test_api_client.py
user: login: kdutia
body: the changes in this file seem to only be on formatting, and the file already passed pre-commit hooks. do we need them?
created_at: 2025-03-20T15:52:02Z
author_association: MEMBER

PR comments:
user: login: linear[bot]
created_at: 2025-03-19T13:07:14Z
body: <p><a href="https://linear.app/climate-policy-radar/issue/SCI-328/productionising-docx-greater-pdf-spike">SCI-328 Productionising docx -&gt; pdf spike</a></p>
author_association: NONE
user: login: mpjuhasz
created_at: 2025-03-20T14:33:54Z
body: > code looks good, and nice refactor! do we want a test for this new functionality too?

@kdutia good point, I've added a test, that tries to imitate what I did manually. It's a bit tricky though, because the libreoffice conversion adds empty pages sometimes, or chunks tabels on multiple pages, slightly changing the content (e.g. adding a bunch of whitespace). To work around this, I've tried some filtering on the content, and then comparing the levenshtein distance of the pdf content between the one that's created by this method, and my conversion from exporting to pdf in pages.
author_association: CONTRIBUTOR