Repository: randombit/botan
Pull request #3028 (2022-08-04 16:00:32)

Test: Allow reporting results as JUnit
Based on the refactoring in #3027 this adds an `XmlReporter` that emits [JUnit5](https://github.com/junit-team/junit5/blob/main/platform-tests/src/test/resources/jenkins-junit.xsd)-ish XML test results. The results are written to disk into a directory specified via `./botan-test --test-results-dir=...` as `Botan-3.0.0-alpha0-tests-<current timestamp>.xml`. "ne can provide additional properties via `--report-properties=key:value,schluessel:wert`.

Eventually, we're planning to visualize the reports into pull requests using external tools, like:
* https://www.check-run-reporter.com/
* https://github.com/mikepenz/action-junit-report

### Example Output

` ./botan-test --test-results-dir=test-results --report-properties=foo:bar block`

```xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="27" failures="0" time="0.268">
<properties>
<property name="CPU flags" value="sse2 ssse3 sse41 sse42 avx2 avx512f avx512dq avx512bw avx512_icelake rdtsc bmi1 bmi2 adx aes_ni clmul rdrand rdseed intel_sha avx512_aes avx512_clmul" />
<property name="current test run" value="1" />
<property name="drbg_seed" value="172A3B0A6E75E755" />
<property name="foo" value="bar" />
<property name="timestamp" value="2022-11-23T14:52:07+0100" />
<property name="total test runs" value="1" />
</properties>
<testsuite name="block" tests="27" failures="0" timestamp="2022-11-23T14:52:07+0100" time="0.256">
<testcase name="AES-128" assertions="17235" timestamp="2022-11-23T14:52:07+0100" time="0.021" file="src/tests/test_block.cpp" line="199" />
<testcase name="AES-192" assertions="20250" timestamp="2022-11-23T14:52:07+0100" time="0.023" file="src/tests/test_block.cpp" line="199" />
<testcase name="AES-256" assertions="23130" timestamp="2022-11-23T14:52:07+0100" time="0.028" file="src/tests/test_block.cpp" line="199" />
<testcase name="ARIA-128" assertions="30" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="ARIA-192" assertions="30" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="ARIA-256" assertions="30" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Blowfish" assertions="930" timestamp="2022-11-23T14:52:07+0100" time="0.007" file="src/tests/test_block.cpp" line="199" />
<testcase name="CAST-128" assertions="720" timestamp="2022-11-23T14:52:07+0100" time="0.001" file="src/tests/test_block.cpp" line="199" />
<testcase name="Camellia-128" assertions="90" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Camellia-192" assertions="45" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Camellia-256" assertions="75" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Cascade(Serpent,AES-256)" assertions="30" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Cascade(Serpent,CAST-128)" assertions="15" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Cascade(Serpent,Twofish)" assertions="45" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="DES" assertions="4815" timestamp="2022-11-23T14:52:07+0100" time="0.004" file="src/tests/test_block.cpp" line="199" />
<testcase name="GOST-28147-89(R3411_94_TestParam)" assertions="270" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="GOST-28147-89(R3411_CryptoPro)" assertions="150" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="IDEA" assertions="16260" timestamp="2022-11-23T14:52:07+0100" time="0.023" file="src/tests/test_block.cpp" line="199" />
<testcase name="Lion(SHA-160,RC4,64)" assertions="15" timestamp="2022-11-23T14:52:07+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Noekeon" assertions="15450" timestamp="2022-11-23T14:52:07+0100" time="0.014" file="src/tests/test_block.cpp" line="199" />
<testcase name="SEED" assertions="60" timestamp="2022-11-23T14:52:08+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="SHACAL2" assertions="61260" timestamp="2022-11-23T14:52:08+0100" time="0.064" file="src/tests/test_block.cpp" line="199" />
<testcase name="SM4" assertions="45" timestamp="2022-11-23T14:52:08+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="Serpent" assertions="47070" timestamp="2022-11-23T14:52:08+0100" time="0.049" file="src/tests/test_block.cpp" line="199" />
<testcase name="Threefish-512" assertions="150" timestamp="2022-11-23T14:52:08+0100" time="0.000" file="src/tests/test_block.cpp" line="199" />
<testcase name="TripleDES" assertions="840" timestamp="2022-11-23T14:52:07+0100" time="0.001" file="src/tests/test_block.cpp" line="199" />
<testcase name="Twofish" assertions="16545" timestamp="2022-11-23T14:52:08+0100" time="0.021" file="src/tests/test_block.cpp" line="199" />
</testsuite>
</testsuites>
```

reneme: I tried [mikepenz/action-junit-report](https://github.com/mikepenz/action-junit-report) and with a few more tweaks this can [annotate failing test cases right into the code](https://github.com/reneme/botan/pull/1/files#diff-452a4a7c46333522991d7f9d22c0b81e58709f553645709d26270792f818ec0e):

![Screenshot 2022-08-05 121419](https://user-images.githubusercontent.com/1562139/183056412-afb4fccb-f937-4470-8357-01f587f7af0e.png)

Obviously, this works only if the failure information contains code locations: I.e. we would need to use some macro-tricks to make this happen. (And touch all the test code to incorporate the macros).

But even without that, it generates test results in the GitHub actions summary view:
![Screenshot 2022-08-05 121751](https://user-images.githubusercontent.com/1562139/183056984-5c7a96a9-3c49-470f-b607-fee6654cc0aa.png)

All of this comes at the expense of using two 3rd party GitHub Actions:
* [mikepenz/action-junit-report](https://github.com/mikepenz/action-junit-report) and
* [dawidd6/action-download-artifact](https://github.com/dawidd6/action-download-artifact)
Plus [an additional workflow YAML](https://github.com/mikepenz/action-junit-report#pr-run-permissions) to orchestrate it all and keep the workflow access permissions in control.
reneme: Now for check-run-reporter.com: This [produces a somewhat more comprehensive report](https://github.com/reneme/botan/pull/2) as a single summary page behind a link in the "Checks"-Box of a Pull Request:

![Screenshot 2022-08-05 172553](https://user-images.githubusercontent.com/1562139/183109788-56add653-ea30-44f0-8247-360b933c3d5a.png)

---

![Screenshot 2022-08-05 171710](https://user-images.githubusercontent.com/1562139/183108016-ad66e87e-9eb2-4cb6-aeea-a1b2e9b550eb.png)

Though, if the same test fails for more than one build configuration it seems to overwrite previous fails, which might or might not be problematic.

On the plus side, it's much easier to set up and can work without a custom 3rd-party action. I.e. one can manually `curl -X POST` the generated JUnit results to the service.

It has a free-tier for open source projects that is capped to 10.000 monthly reports and 1GB of transfer. That may or may not be enough.

Bottom line: neither of these two services is really convincing unfortunately.
@randombit What do you think? Do you see this bringing enough added value?

Frankly: I'm somewhat feeling the urge to write a simple action that is tailored to our needs... Particularly considering other to-be-integrated reports from BoGo, Python-based tests and so on.
codecov-commenter: # [Codecov](https://codecov.io/gh/randombit/botan/pull/3028?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) Report
Base: **92.59**% // Head: **92.58**% // Decreases project coverage by **`-0.00%`** :warning:
> Coverage data is based on head [(`b147d3d`)](https://codecov.io/gh/randombit/botan/pull/3028?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) compared to base [(`307027e`)](https://codecov.io/gh/randombit/botan/commit/307027e40e34a4dab449b381bb550d55d6d7a1d3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd).
> Patch coverage: 89.13% of modified lines in pull request are covered.

> :exclamation: Current head b147d3d differs from pull request most recent head 8720cb5. Consider uploading reports for the commit 8720cb5 to get more accurate results

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #3028      +/-   ##
==========================================
- Coverage   92.59%   92.58%   -0.01%     
==========================================
  Files         599      600       +1     
  Lines       69874    70060     +186     
  Branches     6618     6631      +13     
==========================================
+ Hits        64697    64865     +168     
- Misses       5144     5162      +18     
  Partials       33       33              
```


| [Impacted Files](https://codecov.io/gh/randombit/botan/pull/3028?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) | Coverage Δ | |
|---|---|---|
| [src/tests/runner/test\_runner.h](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3J1bm5lci90ZXN0X3J1bm5lci5o) | `100.00% <ø> (ø)` | |
| [src/tests/runner/test\_reporter.cpp](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3J1bm5lci90ZXN0X3JlcG9ydGVyLmNwcA==) | `87.09% <77.14%> (-12.91%)` | :arrow_down: |
| [src/tests/runner/test\_reporter.h](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3J1bm5lci90ZXN0X3JlcG9ydGVyLmg=) | `85.71% <83.33%> (+55.71%)` | :arrow_up: |
| [src/tests/runner/test\_xml\_reporter.cpp](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3J1bm5lci90ZXN0X3htbF9yZXBvcnRlci5jcHA=) | `91.53% <91.53%> (ø)` | |
| [src/tests/main.cpp](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL21haW4uY3Bw) | `56.81% <100.00%> (+1.00%)` | :arrow_up: |
| [src/tests/runner/test\_runner.cpp](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3J1bm5lci90ZXN0X3J1bm5lci5jcHA=) | `72.26% <100.00%> (+0.41%)` | :arrow_up: |
| [src/tests/tests.cpp](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3Rlc3RzLmNwcA==) | `83.11% <100.00%> (+0.03%)` | :arrow_up: |
| [src/tests/tests.h](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3Rlc3RzLmg=) | `92.55% <100.00%> (+0.46%)` | :arrow_up: |
| [src/fuzzer/mode\_padding.cpp](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2Z1enplci9tb2RlX3BhZGRpbmcuY3Bw) | `93.65% <0.00%> (-6.35%)` | :arrow_down: |
| [src/lib/pubkey/dl\_group/dl\_group.cpp](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2xpYi9wdWJrZXkvZGxfZ3JvdXAvZGxfZ3JvdXAuY3Bw) | `91.22% <0.00%> (-1.06%)` | :arrow_down: |
| ... and [8 more](https://codecov.io/gh/randombit/botan/pull/3028/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/randombit/botan/pull/3028?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd).

reneme: Rebased onto https://github.com/randombit/botan/pull/3058 and added the code locations to the XML report. Hence. we should first review/merge the code locations and then rebase this again onto master.
randombit: I know there are some unresolved questions here wrt reporting actions and how to improve the location reporting, but is there any reason this PR couldn't merge now?
reneme: Indeed there's no show stopper. The JUnit reports just won't be used as is. But we certainly can come back to that later.

Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index c2bbc5495..41db3b2a4 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -56,7 +56,7 @@ jobs:
           arch: ${{ matrix.arch }}
 
       - name: Build and Test Botan
-        run: python3 ./src/scripts/ci_build.py --cc='msvc' --make-tool='jom' --cpu='${{ matrix.arch }}' ${{ matrix.target }}
+        run: python3 ./src/scripts/ci_build.py --cc='msvc' --make-tool='jom' --cpu='${{ matrix.arch }}' --test-results-dir=junit_results ${{ matrix.target }}
 
   linux:
     name: "Linux (Basic)"
@@ -86,7 +86,7 @@ jobs:
           cache-key: linux-${{ matrix.compiler }}-x86_64-${{ matrix.target }}
 
       - name: Build and Test Botan
-        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' ${{ matrix.target }}
+        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' --test-results-dir=junit_results ${{ matrix.target }}
 
   macos:
     name: "macOS (Basic)"
@@ -119,7 +119,7 @@ jobs:
           cache-key: macos-${{ matrix.compiler }}-x86_64-${{ matrix.target }}
 
       - name: Build and Test Botan
-        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' ${{ matrix.target }}
+        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' --test-results-dir=junit_results ${{ matrix.target }}
 
   specials:
     name: "Metrics and Specials"
@@ -176,7 +176,7 @@ jobs:
           cache-key: ${{ matrix.host_os }}-${{ matrix.compiler }}-x86_64-${{ matrix.target }}
 
       - name: Build and Test Botan
-        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' --make-tool='${{ matrix.make_tool }}' ${{ matrix.target }}
+        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' --make-tool='${{ matrix.make_tool }}' --test-results-dir=junit_results ${{ matrix.target }}
 
   x-compile:
     name: "Cross-Compilation"
@@ -222,4 +222,4 @@ jobs:
           cache-key: ${{ matrix.host_os }}-${{ matrix.compiler }}-xcompile-${{ matrix.target }}
 
       - name: Build and Test Botan
-        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' ${{ matrix.target }}
+        run: python3 ./src/scripts/ci_build.py --cc='${{ matrix.compiler }}' --test-results-dir=junit_results ${{ matrix.target }}
diff --git a/configure.py b/configure.py
index c26c3837c..b45f5087b 100755
--- a/configure.py
+++ b/configure.py
@@ -806,8 +806,6 @@ class InfoObject:
                     self.parent_module = os.path.basename(obj_dir)
                     break
                 (obj_dir, _) = os.path.split(obj_dir)
-                if obj_dir == '/':
-                    obj_dir = None
         else:
             self.basename = basename.replace('.txt', '')
 
@@ -1821,12 +1819,13 @@ def yield_objectfile_list(sources, obj_dir, obj_suffix, options):
 
     for src in sources:
         (directory, filename) = os.path.split(os.path.normpath(src))
-        parts_in_src = directory.split('src' + os.sep)
-        parts = []
+        parts = directory.split(os.sep)
 
-        if len(parts_in_src) > 1:
-            parts = (parts_in_src[-1].split(os.sep))[1:]
-        elif not options.amalgamation or filename.find(options.name_amalgamation) == -1:
+        if 'src' in parts:
+            parts = parts[parts.index('src')+2:]
+        elif options.amalgamation and filename.find(options.name_amalgamation) != -1:
+            parts = []
+        else:
             raise InternalError("Unexpected file '%s/%s'" % (directory, filename))
 
         if parts != []:
diff --git a/doc/security.rst b/doc/security.rst
index 11ed81dbd..57ef764d3 100644
--- a/doc/security.rst
+++ b/doc/security.rst
@@ -15,30 +15,6 @@ mail please use::
 This key can be found in the file ``doc/pgpkey.txt`` or online at
 https://keybase.io/jacklloyd and on most PGP keyservers.
 
-2022
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-* 2022-11-16: Failure to correctly check OCSP responder embedded certificate
-
-  OCSP responses for some end entity are either signed by the issuing CA certificate of
-  the PKI, or an OCSP responder certificate that the PKI authorized to sign responses in
-  their name. In the latter case, the responder certificate (and its validation path
-  certificate) may be embedded into the OCSP response and clients must verify that such
-  certificates are indeed authorized by the CA when validating OCSP responses.
-
-  The OCSP implementation failed to verify that an authorized responder certificate
-  embedded in an OCSP response is authorized by the issuing CA. As a result, any valid
-  signature by an embedded certificate passed the check and was allowed to make claims
-  about the revocation status of certificates of any CA.
-
-  Attackers that are in a position to spoof OCSP responses for a client could therefore
-  render legitimate certificates of a 3rd party CA as revoked or even use a compromised
-  (and actually revoked) certificate by spoofing an OCSP-"OK" response. E.g. an attacker
-  could exploit this to impersonate a legitimate TLS server using a compromised
-  certificate of that host and get around the revocation check using OCSP stapling.
-
-  Introduced in 1.11.34, fixed in 2.19.3
-
 2020
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
diff --git a/news.rst b/news.rst
index d57f2b20b..9993bd171 100644
--- a/news.rst
+++ b/news.rst
@@ -143,13 +143,6 @@ Version 3.0.0, Not Yet Released
 
 * Generate a ``compile_commands.json`` for use with Clang tooling.
 
-Version 2.19.3, 2022-11-16
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-* CVE-2022-43705: A malicious OCSP responder could forge OCSP
-  responses due to a failure to validate that an embedded certificate
-  was issued by the end-entity issuing certificate authority.
-
 Version 2.19.2, 2022-06-03
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
diff --git a/readme.rst b/readme.rst
index d5f374e61..8dcd4f532 100644
--- a/readme.rst
+++ b/readme.rst
@@ -27,9 +27,9 @@ If you think you have found a security issue, see the `security page
 <https://botan.randombit.net/security.html>`_ for contact information.
 
 The latest release is
-`2.19.3 <https://botan.randombit.net/releases/Botan-2.19.3.tar.xz>`_
-`(sig) <https://botan.randombit.net/releases/Botan-2.19.3.tar.xz.asc>`_,
-released on 2022-11-16.
+`2.19.2 <https://botan.randombit.net/releases/Botan-2.19.2.tar.xz>`_
+`(sig) <https://botan.randombit.net/releases/Botan-2.19.2.tar.xz.asc>`_,
+released on 2022-06-03.
 All releases are signed with a `PGP key <https://botan.randombit.net/pgpkey.txt>`_.
 See the `release notes <https://botan.randombit.net/news.html>`_ for
 what is new. Botan is also available through most
diff --git a/src/scripts/ci_build.py b/src/scripts/ci_build.py
index 850799922..8b443863c 100755
--- a/src/scripts/ci_build.py
+++ b/src/scripts/ci_build.py
@@ -76,7 +76,7 @@ def build_targets(target, target_os):
         yield 'bogo_shim'
 
 def determine_flags(target, target_os, target_cpu, target_cc, cc_bin,
-                    ccache, root_dir, pkcs11_lib, use_gdb, disable_werror, extra_cxxflags,
+                    ccache, root_dir, test_results_dir, pkcs11_lib, use_gdb, disable_werror, extra_cxxflags,
                     disabled_tests):
     # pylint: disable=too-many-branches,too-many-statements,too-many-arguments,too-many-locals
 
@@ -110,6 +110,20 @@ def determine_flags(target, target_os, target_cpu, target_cc, cc_bin,
     test_prefix = []
     test_cmd = [os.path.join(root_dir, 'botan-test')]
 
+    # generate JUnit test report
+    if test_results_dir:
+        if not os.path.isdir(test_results_dir):
+            raise Exception("Test results directory does not exist")
+
+        def sanitize_kv(some_string):
+            return some_string.replace(':', '').replace(',', '')
+
+        report_props = {"ci_target": target, "os": target_os}
+
+        test_cmd += ['--test-results-dir=%s' % test_results_dir]
+        test_cmd += ['--report-properties=%s' %
+                     ','.join(['%s:%s' % (sanitize_kv(k), sanitize_kv(v)) for k, v in report_props.items()])]
+
     install_prefix = tempfile.mkdtemp(prefix='botan-install-')
 
     flags = ['--prefix=%s' % (install_prefix),
@@ -424,6 +438,9 @@ def parse_args(args):
     parser.add_option('--run-under-gdb', dest='use_gdb', action='store_true', default=False,
                       help='Run test suite under gdb and capture backtrace')
 
+    parser.add_option('--test-results-dir', default=None,
+                      help='Directory to store JUnit XML test reports')
+
     return parser.parse_args(args)
 
 def have_prog(prog):
@@ -527,9 +544,12 @@ def main(args=None):
         cmds.append([py_interp, '-m', 'pylint'] + pylint_flags + full_paths)
 
     else:
+        if options.test_results_dir:
+            os.makedirs(options.test_results_dir)
+
         config_flags, run_test_command, make_prefix, install_prefix = determine_flags(
             target, options.os, options.cpu, options.cc,
-            options.cc_bin, options.compiler_cache, root_dir,
+            options.cc_bin, options.compiler_cache, root_dir, options.test_results_dir,
             options.pkcs11_lib, options.use_gdb, options.disable_werror,
             options.extra_cxxflags, options.disabled_tests)
 


Test patch:
diff --git a/src/tests/main.cpp b/src/tests/main.cpp
index 3b1a6e074..26e30577a 100644
--- a/src/tests/main.cpp
+++ b/src/tests/main.cpp
@@ -57,8 +57,8 @@ int main(int argc, char* argv[])
       const std::string arg_spec =
          "botan-test --verbose --help --data-dir= --pkcs11-lib= --provider= "
          "--log-success --abort-on-first-fail --no-avoid-undefined --skip-tests= "
-         "--test-threads=0 --run-long-tests --run-online-tests --test-runs=1 --drbg-seed= "
-         "*suites";
+         "--test-threads=0 --test-results-dir= --run-long-tests --run-online-tests "
+         "--test-runs=1 --drbg-seed= --report-properties= *suites";
 
       Botan_CLI::Argument_Parser parser(arg_spec);
 
@@ -88,6 +88,8 @@ int main(int argc, char* argv[])
          parser.get_arg("pkcs11-lib"),
          parser.get_arg("provider"),
          parser.get_arg("drbg-seed"),
+         parser.get_arg("test-results-dir"),
+         parser.get_arg_list("report-properties"),
          parser.get_arg_sz("test-runs"),
          parser.get_arg_sz("test-threads"),
          parser.flag_set("verbose"),
diff --git a/src/tests/runner/test_reporter.cpp b/src/tests/runner/test_reporter.cpp
index 44785fdb2..b42fc49f3 100644
--- a/src/tests/runner/test_reporter.cpp
+++ b/src/tests/runner/test_reporter.cpp
@@ -11,6 +11,70 @@
 
 namespace Botan_Tests {
 
+namespace {
+
+template <typename T>
+constexpr std::optional<T>
+operator+(const std::optional<T>& a, const std::optional<T>& b)
+   {
+   if(!a.has_value() || !b.has_value())
+      {
+      return std::nullopt;
+      }
+
+   return a.value() + b.value();
+   }
+
+}
+
+TestSummary::TestSummary(const Test::Result& result)
+   : name(result.who())
+   , code_location(result.code_location())
+   , assertions(result.tests_run())
+   , notes(result.notes())
+   , failures(result.failures())
+   , timestamp(result.timestamp())
+   , elapsed_time(result.elapsed_time()) {}
+
+
+Testsuite::Testsuite(std::string name) : m_name(std::move(name)) {}
+
+void Testsuite::record(const Test::Result& result)
+   {
+   m_results.emplace_back(result);
+   }
+
+size_t Testsuite::tests_passed() const
+   {
+   return std::count_if(m_results.begin(), m_results.end(),
+                        [](const auto& r) { return r.passed(); });
+   }
+
+size_t Testsuite::tests_failed() const
+   {
+   return std::count_if(m_results.begin(), m_results.end(),
+                        [](const auto& r) { return r.failed(); });
+   }
+
+std::chrono::system_clock::time_point Testsuite::timestamp() const
+   {
+   return std::transform_reduce(
+      m_results.begin(), m_results.end(),
+      std::chrono::system_clock::time_point::max(),
+            [](const auto& a, const auto& b) { return std::min(a, b); },
+            [](const auto& result) { return result.timestamp; });
+   }
+
+std::optional<std::chrono::nanoseconds> Testsuite::elapsed_time() const
+   {
+   return std::transform_reduce(
+      m_results.begin(), m_results.end(),
+      std::make_optional(std::chrono::nanoseconds::zero()),
+      [](const auto& a, const auto& b) { return a + b; },
+      [](const auto& result) { return result.elapsed_time; });
+   }
+
+
 Reporter::Reporter(const Test_Options& opts)
    : m_total_test_runs(opts.test_runs())
    , m_current_test_run(0)
@@ -25,10 +89,17 @@ void Reporter::next_test_run()
    {
    m_start_time = std::chrono::high_resolution_clock::now();
    ++m_current_test_run;
+   m_testsuites.clear();
 
    next_run();
    }
 
+void Reporter::record(const std::string& name, const Test::Result& result)
+   {
+   auto& suite = m_testsuites.try_emplace(name, name).first->second;
+   suite.record(result);
+   }
+
 void Reporter::record(const std::string& testsuite_name,
                       const std::vector<Botan_Tests::Test::Result>& results)
    {
@@ -53,6 +124,31 @@ void Reporter::record(const std::string& testsuite_name,
       }
    }
 
+
+size_t Reporter::tests_run() const
+   {
+   return std::transform_reduce(
+      m_testsuites.begin(), m_testsuites.end(),
+      size_t(0), std::plus{},
+      [](const auto& testsuite) { return testsuite.second.tests_run(); });
+   }
+
+size_t Reporter::tests_passed() const
+   {
+   return std::transform_reduce(
+      m_testsuites.begin(), m_testsuites.end(),
+      size_t(0), std::plus{},
+      [](const auto& testsuite) { return testsuite.second.tests_passed(); });
+   }
+
+size_t Reporter::tests_failed() const
+   {
+   return std::transform_reduce(
+      m_testsuites.begin(), m_testsuites.end(),
+      size_t(0), std::plus{},
+      [](const auto& testsuite) { return testsuite.second.tests_failed(); });
+   }
+
 std::chrono::nanoseconds Reporter::elapsed_time() const
    {
    return std::chrono::high_resolution_clock::now() - m_start_time;
diff --git a/src/tests/runner/test_reporter.h b/src/tests/runner/test_reporter.h
index de2b1a8d9..72917ba6e 100644
--- a/src/tests/runner/test_reporter.h
+++ b/src/tests/runner/test_reporter.h
@@ -12,6 +12,59 @@
 
 namespace Botan_Tests {
 
+/**
+ * Summary data holder for an individual test (i.e. one `Test::Result` instance)
+ */
+class TestSummary final
+   {
+   public:
+      TestSummary(const Test::Result& result);
+
+      bool passed() const { return failures.empty(); }
+      bool failed() const { return !failures.empty(); }
+
+   public:
+      const std::string name;
+      const std::optional<CodeLocation> code_location;
+
+      const size_t assertions;
+      const std::vector<std::string> notes;
+      const std::vector<std::string> failures;
+
+      const std::chrono::system_clock::time_point timestamp;
+      const std::optional<std::chrono::nanoseconds> elapsed_time;
+   };
+
+
+/**
+ * Summary data holder for a test suite containing potentially many test cases
+ */
+class Testsuite final
+   {
+   public:
+      Testsuite(std::string name);
+
+      void record(const Test::Result& result);
+
+      size_t tests_run() const { return m_results.size(); }
+      size_t tests_passed() const;
+      size_t tests_failed() const;
+
+      /// Returns the oldest time stamp in all contained test cases
+      std::chrono::system_clock::time_point timestamp() const;
+
+      /// Returns the cumulative elapsed time of all contained test cases
+      std::optional<std::chrono::nanoseconds> elapsed_time() const;
+
+      const std::string& name() const { return m_name; }
+      const std::vector<TestSummary>& results() const { return m_results; }
+
+   private:
+      const std::string        m_name;
+      std::vector<TestSummary> m_results;
+   };
+
+
 /**
  * @brief Base class for Botan's test result reporting facility
  *
@@ -20,6 +73,7 @@ namespace Botan_Tests {
 class Reporter
    {
    public:
+      using TestsuiteMap = std::map<std::string, Testsuite>;
       using PropertyMap = std::map<std::string, std::string>;
 
    public:
@@ -59,17 +113,19 @@ class Reporter
       /**
        * @brief Reports a single test result
        *
-       * The default implementation doesn't do any reporting.
+       * The default implementation records the result as `Testsuite` and
+       * `TestSummary` objects but doesn't do any reporting.
        *
-       * Subclasses should override this method to add custom handling.
+       * Subclasses should override this method to add custom handling and/or
+       * replace the test summary recording entirely. Note that the protected
+       * accessor methods won't work as expected if this method isn't up-called.
        *
-       * @param name    The name of the test case the result was created
+       * @param name    The name of the test case the result was created in
+       *                Note that multiple results can be reported under the
+       *                same name. The `Reporter` will coalesce those.
        * @param result  The test result data structure of a finished test case
        */
-      virtual void record(const std::string& name, const Test::Result& result)
-         {
-         BOTAN_UNUSED(name, result);
-         }
+      virtual void record(const std::string& name, const Test::Result& result);
 
       /**
        * @brief Reports a list of test results for a given test suite
@@ -104,9 +160,14 @@ class Reporter
        */
       virtual void next_run() = 0;
 
+      size_t tests_run() const;
+      size_t tests_passed() const;
+      size_t tests_failed() const;
+
       std::chrono::nanoseconds elapsed_time() const;
 
       const PropertyMap& properties() const { return m_properties; }
+      const TestsuiteMap& testsuites() const { return m_testsuites; }
 
       size_t current_test_run() const { return m_current_test_run; }
       size_t total_test_runs() const { return m_total_test_runs; }
@@ -117,6 +178,7 @@ class Reporter
 
       PropertyMap m_properties;
 
+      TestsuiteMap m_testsuites;
       std::chrono::high_resolution_clock::time_point m_start_time;
    };
 
diff --git a/src/tests/runner/test_runner.cpp b/src/tests/runner/test_runner.cpp
index 1b931b958..cb22c0b0f 100644
--- a/src/tests/runner/test_runner.cpp
+++ b/src/tests/runner/test_runner.cpp
@@ -1,5 +1,6 @@
 /*
 * (C) 2017 Jack Lloyd
+* (C) 2022 René Meusel, Rohde & Schwarz Cybersecurity
 *
 * Botan is released under the Simplified BSD License (see license.txt)
 */
@@ -8,6 +9,7 @@
 
 #include "test_runner.h"
 #include "test_stdout_reporter.h"
+#include "test_xml_reporter.h"
 
 #include <botan/version.h>
 #include <botan/internal/loadstor.h>
@@ -89,6 +91,14 @@ bool Test_Runner::run(const Test_Options& opts)
    const std::set<std::string>& to_skip = opts.skip_tests();
 
    m_reporters.emplace_back(std::make_unique<StdoutReporter>(opts, output()));
+   if(!opts.xml_results_dir().empty())
+      {
+#if defined(BOTAN_TARGET_OS_HAS_FILESYSTEM)
+      m_reporters.emplace_back(std::make_unique<XmlReporter>(opts, opts.xml_results_dir()));
+#else
+      output() << "Generating test report files is not supported on this platform\n";
+#endif
+      }
 
    if(req.empty())
       {
diff --git a/src/tests/runner/test_runner.h b/src/tests/runner/test_runner.h
index ced63171e..ed2bbf12a 100644
--- a/src/tests/runner/test_runner.h
+++ b/src/tests/runner/test_runner.h
@@ -1,5 +1,6 @@
 /*
 * (C) 2017 Jack Lloyd
+* (C) 2022 René Meusel, Rohde & Schwarz Cybersecurity
 *
 * Botan is released under the Simplified BSD License (see license.txt)
 */
diff --git a/src/tests/runner/test_xml_reporter.cpp b/src/tests/runner/test_xml_reporter.cpp
new file mode 100644
index 000000000..c729bcc6e
--- /dev/null
+++ b/src/tests/runner/test_xml_reporter.cpp
@@ -0,0 +1,299 @@
+/*
+* (C) 2022 Jack Lloyd
+* (C) 2022 René Meusel, Rohde & Schwarz Cybersecurity
+*
+* Botan is released under the Simplified BSD License (see license.txt)
+*/
+
+#include "test_xml_reporter.h"
+
+#if defined(BOTAN_TARGET_OS_HAS_FILESYSTEM)
+
+#include <botan/internal/loadstor.h>
+#include <botan/version.h>
+
+#include <iomanip>
+#include <numeric>
+#include <time.h>
+
+namespace Botan_Tests {
+
+namespace {
+
+std::tm* localtime(const time_t* timer, std::tm *buffer)
+   {
+#if defined(BOTAN_BUILD_COMPILER_IS_MSVC) || \
+    defined(BOTAN_TARGET_OS_IS_MINGW)     || \
+    defined(BOTAN_TARGET_OS_IS_CYGWIN)    || \
+    defined(BOTAN_TARGET_OS_IS_WINDOWS)
+   localtime_s(buffer, timer);
+#else
+   localtime_r(timer, buffer);
+#endif
+   return buffer;
+   }
+
+/// formats a given time point in ISO 8601 format (with time zone)
+std::string format(const std::chrono::system_clock::time_point& tp)
+   {
+   auto seconds_since_epoch = std::chrono::system_clock::to_time_t(tp);
+
+   std::ostringstream out;
+   std::tm buffer{};
+   out << std::put_time(localtime(&seconds_since_epoch, &buffer), "%FT%T%z");
+   return out.str();
+   }
+
+std::string format(const std::chrono::nanoseconds& dur)
+   {
+   const float secs = static_cast<float>(dur.count()) / 1000000000;
+
+   std::ostringstream out;
+   out.precision(3);
+   out << std::fixed << secs;
+   return out.str();
+   }
+
+}
+
+XmlReporter::XmlReporter(const Test_Options& opts, std::string output_dir)
+   : Reporter(opts)
+   , m_output_dir(std::move(output_dir))
+   {
+   set_property("timestamp", format(std::chrono::system_clock::now()));
+   auto custom_props = opts.report_properties();
+   for(const auto& prop : custom_props)
+      {
+      set_property(prop.first, prop.second);
+      }
+   }
+
+void XmlReporter::render() const
+   {
+   BOTAN_STATE_CHECK(m_outfile.has_value() && m_outfile->good());
+
+   render_preamble(m_outfile.value());
+   render_testsuites(m_outfile.value());
+   }
+
+std::string XmlReporter::get_unique_output_filename() const
+   {
+   const uint64_t ts = Botan_Tests::Test::timestamp();
+   std::vector<uint8_t> seed(8);
+   Botan::store_be(ts, seed.data());
+
+   std::stringstream ss;
+   ss << m_output_dir << "/"
+      << "Botan-"
+      << Botan::short_version_string()
+      << "-tests-"
+      << Botan::hex_encode(seed, false)
+      << ".xml";
+
+   return ss.str();
+   }
+
+void XmlReporter::next_run()
+   {
+   if(m_outfile.has_value())
+      {
+      m_outfile.reset();
+      }
+
+   set_property("current test run", std::to_string(current_test_run()));
+   set_property("total test runs", std::to_string(total_test_runs()));
+   const auto file = get_unique_output_filename();
+   m_outfile = std::ofstream(file, std::ofstream::out | std::ofstream::trunc);
+
+   if(!m_outfile->good())
+      {
+      std::stringstream ss;
+      ss << "Failed to open '" << file << "' for writing JUnit report.";
+      throw Botan::System_Error(ss.str());
+      }
+   }
+
+
+// == == == == == == == == == == == == == == == == == == == == == == == == == ==
+// XML Rendering
+// == == == == == == == == == == == == == == == == == == == == == == == == == ==
+
+
+namespace {
+
+void replace(std::string& str, const std::string& from, const std::string& to)
+   {
+   if(from.empty())
+      { return; }
+
+   for(size_t offset = 0, pos = 0;
+       (pos = str.find(from, offset)) != std::string::npos;
+       offset = pos + to.size())
+      {
+      str.replace(pos, from.size(), to);
+      }
+   }
+
+std::string escape(std::string str)
+   {
+   replace(str, "&",  "&amp;");
+   replace(str, "<",  "&lt;");
+   replace(str, ">",  "&gt;");
+   replace(str, "\"", "&quot;");
+   replace(str, "'",  "&apos;");
+   return str;
+   }
+
+std::string format_cdata(std::string str)
+   {
+   // XML CDATA payloads are not evaluated, hence no special character encoding
+   // is needed.
+   // Though the termination sequence (i.e. ']]>') must not appear in
+   // a CDATA payload frame. The only way to escape it is to terminate the CDATA
+   // sequence and break the payload's termination sequence into the adjacent
+   // CDATA frames.
+   //
+   //   See: https://stackoverflow.com/a/223782
+   replace(str, "]]>", "]]]><![CDATA[]>");
+   //            ^^^ -> ^~~~~~~~~~~~~^^
+
+   // wrap the (escaped) payload into a CDATA frame
+   std::ostringstream out;
+   out << "<![CDATA["
+       << str
+       << "]]>";
+   return out.str();
+   }
+
+}
+
+void XmlReporter::render_preamble(std::ostream& out) const
+   {
+   out << "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
+   }
+
+void XmlReporter::render_properties(std::ostream& out) const
+   {
+   if(properties().empty())
+      {
+      return;
+      }
+
+   out << "<properties>\n";
+   for(const auto& prop : properties())
+      {
+      out << "<property"
+          << " name=\"" << escape(prop.first) << "\""
+          << " value=\"" << escape(prop.second) << "\""
+          << " />\n";
+      }
+   out << "</properties>\n";
+   }
+
+void XmlReporter::render_testsuites(std::ostream& out) const
+   {
+   // render an empty testsuites tag even if no tests were run
+   out << "<testsuites"
+       << " tests=\"" << tests_run() << "\""
+       << " failures=\"" << tests_failed() << "\""
+       << " time=\"" << format(elapsed_time()) << "\">\n";
+
+   // Note: In the JUnit .xsd spec, <properties> appear only in individual
+   //       test cases. This deviation from the spec allows us to embed
+   //       specific platform information about this particular test run.
+   render_properties(out);
+
+   for(const auto& suite : testsuites())
+      {
+      render_testsuite(out, suite.second);
+      }
+
+   out << "</testsuites>\n";
+   }
+
+void XmlReporter::render_testsuite(std::ostream& out, const Testsuite& suite) const
+   {
+   out << "<testsuite"
+       << " name=\"" << escape(suite.name()) << "\""
+       << " tests=\"" << suite.tests_run() << "\""
+       << " failures=\"" << suite.tests_failed() << "\""
+       << " timestamp=\"" << format(suite.timestamp()) << "\"";
+
+   const auto elapsed = suite.elapsed_time();
+   if(elapsed.has_value())
+      {
+      out << " time=\"" << format(elapsed.value()) << "\"";
+      }
+
+   if(suite.results().empty())
+      {
+      out << " />\n";
+      }
+   else
+      {
+      out << ">\n";
+
+      for(const auto& result : suite.results())
+         {
+         render_testcase(out, result);
+         }
+
+      out << "</testsuite>\n";
+      }
+   }
+
+void XmlReporter::render_testcase(std::ostream& out, const TestSummary& test) const
+   {
+   out << "<testcase"
+      << " name=\"" << escape(test.name) << "\""
+      << " assertions=\"" << test.assertions << "\""
+      << " timestamp=\"" << format(test.timestamp) << "\"";
+
+   if(test.elapsed_time.has_value())
+      {
+      out << " time=\"" << format(test.elapsed_time.value()) << "\"";
+      }
+
+   if(test.code_location.has_value())
+      {
+      out << " file=\"" << escape(test.code_location->path) << "\""
+          << " line=\"" << test.code_location->line << "\"";
+      }
+
+   if(test.failures.empty() && test.notes.empty())
+      {
+      out << " />\n";
+      }
+   else
+      {
+      out << ">\n";
+      render_failures_and_stdout(out, test);
+      out << "</testcase>\n";
+      }
+   }
+
+void XmlReporter::render_failures_and_stdout(std::ostream& out, const TestSummary& test) const
+   {
+   for(const auto& failure : test.failures)
+      {
+      out << "<failure>\n"
+          << format_cdata(failure) << "\n"
+          << "</failure>\n";
+      }
+
+   // xUnit format does not have a special tag for test notes, hence we
+   // render it into the freetext 'system-out'
+   if(!test.notes.empty())
+      {
+      out << "<system-out>\n";
+      for(const auto& note : test.notes)
+         {
+         out << format_cdata(note) << '\n';
+         }
+      out << "</system-out>\n";
+      }
+   }
+
+}
+
+#endif  // defined(BOTAN_TARGET_OS_HAS_FILESYSTEM)
diff --git a/src/tests/runner/test_xml_reporter.h b/src/tests/runner/test_xml_reporter.h
new file mode 100644
index 000000000..d6583b190
--- /dev/null
+++ b/src/tests/runner/test_xml_reporter.h
@@ -0,0 +1,56 @@
+/*
+* (C) 2022 Jack Lloyd
+* (C) 2022 René Meusel, Rohde & Schwarz Cybersecurity
+*
+* Botan is released under the Simplified BSD License (see license.txt)
+*/
+
+#ifndef BOTAN_TEST_XML_REPORTER_H_
+#define BOTAN_TEST_XML_REPORTER_H_
+
+#include <botan/build.h>
+#if defined(BOTAN_TARGET_OS_HAS_FILESYSTEM)
+
+#include "test_reporter.h"
+
+#include <optional>
+#include <fstream>
+
+namespace Botan_Tests {
+
+/**
+ * XML JUnit exporter for test results
+ *
+ * JUnit schema follows:
+ *   https://github.com/junit-team/junit5/blob/main/platform-tests/src/test/resources/jenkins-junit.xsd
+ */
+class XmlReporter : public Reporter
+   {
+   public:
+      XmlReporter(const Test_Options& opts, std::string output_dir);
+
+      void render() const override;
+
+      void next_run() override;
+
+   private:
+      std::string get_unique_output_filename() const;
+
+      void render_preamble(std::ostream& out) const;
+      void render_properties(std::ostream& out) const;
+      void render_testsuites(std::ostream& out) const;
+      void render_testsuite(std::ostream& out, const Testsuite& suite) const;
+      void render_testcase(std::ostream& out, const TestSummary& test) const;
+      void render_failures_and_stdout(std::ostream& out, const TestSummary& test) const;
+
+   private:
+      std::string m_output_dir;
+
+      mutable std::optional<std::ofstream> m_outfile;
+   };
+
+}
+
+#endif  // defined(BOTAN_TARGET_OS_HAS_FILESYSTEM)
+
+#endif  // BOTAN_TEST_XML_REPORTER_H_
diff --git a/src/tests/tests.cpp b/src/tests/tests.cpp
index 827c70d18..4a6c5a634 100644
--- a/src/tests/tests.cpp
+++ b/src/tests/tests.cpp
@@ -46,6 +46,7 @@ void Test::Result::merge(const Result& other, bool ignore_test_name)
       m_where = other.m_where;
       }
 
+   m_timestamp = std::min(m_timestamp, other.m_timestamp);
    m_ns_taken += other.m_ns_taken;
    m_tests_passed += other.m_tests_passed;
    m_fail_log.insert(m_fail_log.end(), other.m_fail_log.begin(), other.m_fail_log.end());
@@ -1180,4 +1181,24 @@ std::vector<Test::Result> Text_Based_Test::run()
    return results;
    }
 
+std::map<std::string, std::string> Test_Options::report_properties() const
+   {
+   std::map<std::string, std::string> result;
+
+   for(const auto& prop : m_report_properties)
+      {
+      const auto colon = prop.find(':');
+      // props without a colon separator or without a name are not allowed
+      if(colon == std::string::npos || colon == 0)
+         {
+         throw Test_Error("--report-properties should be of the form <key>:<value>,<key>:<value>,...");
+         }
+
+      result.insert_or_assign(prop.substr(0, colon),
+                              prop.substr(colon + 1, prop.size() - colon - 1));
+      }
+
+   return result;
+   }
+
 }
diff --git a/src/tests/tests.h b/src/tests/tests.h
index 98a7a07ce..2b0b7701f 100644
--- a/src/tests/tests.h
+++ b/src/tests/tests.h
@@ -67,6 +67,8 @@ class Test_Options
                    const std::string& pkcs11_lib,
                    const std::string& provider,
                    const std::string& drbg_seed,
+                   const std::string& xml_results_dir,
+                   const std::vector<std::string>& report_properties,
                    size_t test_runs,
                    size_t test_threads,
                    bool verbose,
@@ -80,6 +82,8 @@ class Test_Options
          m_pkcs11_lib(pkcs11_lib),
          m_provider(provider),
          m_drbg_seed(drbg_seed),
+         m_xml_results_dir(xml_results_dir),
+         m_report_properties(report_properties),
          m_test_runs(test_runs),
          m_test_threads(test_threads),
          m_verbose(verbose),
@@ -104,6 +108,10 @@ class Test_Options
 
       const std::string& drbg_seed() const { return m_drbg_seed; }
 
+      const std::string& xml_results_dir() const { return m_xml_results_dir; }
+
+      std::map<std::string, std::string> report_properties() const;
+
       size_t test_runs() const { return m_test_runs; }
 
       size_t test_threads() const { return m_test_threads; }
@@ -125,6 +133,8 @@ class Test_Options
       std::string m_pkcs11_lib;
       std::string m_provider;
       std::string m_drbg_seed;
+      std::string m_xml_results_dir;
+      std::vector<std::string> m_report_properties;
       size_t m_test_runs;
       size_t m_test_threads;
       bool m_verbose;
@@ -160,7 +170,9 @@ class Test
       class Result final
          {
          public:
-            explicit Result(std::string who) : m_who(std::move(who)) {}
+            explicit Result(std::string who)
+               : m_who(std::move(who))
+               , m_timestamp(std::chrono::system_clock::now()) {}
 
             /**
              * This 'consolidation constructor' creates a single test result from
@@ -190,6 +202,9 @@ class Test
                return m_who;
                }
 
+            const std::vector<std::string>& failures() const { return m_fail_log; }
+            const std::vector<std::string>& notes() const { return m_log; }
+
             std::optional<std::chrono::nanoseconds> elapsed_time() const
                {
                if (m_ns_taken == 0)
@@ -202,6 +217,11 @@ class Test
                   }
                }
 
+            const std::chrono::system_clock::time_point& timestamp() const
+               {
+               return m_timestamp;
+               }
+
             std::string result_string() const;
 
             static Result Failure(const std::string& who,
@@ -545,6 +565,7 @@ class Test
          private:
             std::string m_who;
             std::optional<CodeLocation> m_where;
+            std::chrono::system_clock::time_point m_timestamp;
             uint64_t m_started = 0;
             uint64_t m_ns_taken = 0;
             size_t m_tests_passed = 0;