Repository: buildtesters/buildtest (language: python)

Issue:
[Bug]: spack test results with suite name not expected 
### Bug Description

the problem is best described in test result https://my.cdash.org/test/67729705?graph=status 

The test failure reports

```console
Lmod has detected the following error: Swap failed: "PrgEnv-nvidia" is not
loaded.
==> Error: Test suite "spack_test_hdf5_e4s_21.11" already exists. Try another alias.
```

for this buildspec

```yaml
buildspecs:
  spack_test_hdf5_e4s_21.11:
    type: spack
    executor: '(perlmutter|muller).local.sh'
    description: "Test hdf5@1.12.1 for e4s/21.11 stack for Perlmutter via spack test"
    tags: e4s
    pre_cmds: |
      module load e4s/21.11
      module swap PrgEnv-nvidia PrgEnv-gnu
    spack:
      root: /global/common/software/spackecp/perlmutter/e4s-21.11/spack/
      verify_spack: false
      test:
        run:
          specs: ['hdf5@1.12.1']
        results:
          option: '-l'
          specs: ['hdf5@1.12.1']
``` 

The issue is that we auto-generate the suite name and since the suite name is taken when rerunning the test this test never ran.



### Steps to reproduce the error

Just run `buildtest build` against any spec in this directory https://github.com/buildtesters/buildtest-nersc/tree/devel/buildspecs/e4s/spack_test/perlmutter

### Version and HEAD commit

```shell
devel branch
```


### Relevant log output

_No response_

### Post question in Slack

- [X] I agree that I posted my question in [slack](https://hpcbuildtest.slack.com/) before creating this issue

### Is there an existing issue

- [X] I confirm there is no existing [issue](https://github.com/buildtesters/buildtest/issues) for this issue


Discussion / hints:
this is related to issue https://github.com/buildtesters/buildtest-nersc/issues/157.

@wspear suggested we can auto-generate alias names for every test one idea could be to remove the `suite` property under `results` such as this test defined in https://buildtest.readthedocs.io/en/devel/buildspecs/spack.html#spack-test

```yaml
buildspecs:
  spack_test_m4:
    type: spack
    executor: generic.local.bash
    description: "Run spack test for m4 package and report results"
    tags: [spack]
    spack:
      root: $HOME/spack
      test:
        run:
          specs: ['m4']
        results:
          suite: ['spack_test_m4']
          option: "-l"
```

The suite name is name of test `spack_test_m4` so if the key `spec` is not defined under results we retrieve the suite name otherwise use the `specs`.

The question is whether we give user the ability to specify the suite name under `spack test run --alias <alias_name>` they must do this under the `run` section using `options` so it is doable and they can do the following

```yaml
buildspecs:
  spack_test_m4:
    type: spack
    executor: generic.local.bash
    description: "Run spack test for m4 package and report results"
    tags: [spack]
    spack:
      root: $HOME/spack
      test:
        run:
          specs: ['m4']
          options: "--alias m4_test"
        results:
          suite: ['m4_test']
          option: "-l"
```

However i think that it might not be useful having `suite` property this only makes sense if user specifies the `options` property under `run` otherwise they wont know the alias names of hand since spack auto-generates them at runtime. Given that this becomes a problem when re-running the same test multiple times, the command `spack test run --alias` will fail if suite name is already present.


Fix patch:
diff --git a/docs/buildspecs/spack.rst b/docs/buildspecs/spack.rst
index ab4b71c1e..2fe473323 100644
--- a/docs/buildspecs/spack.rst
+++ b/docs/buildspecs/spack.rst
@@ -248,40 +248,41 @@ Spack Test
 
 .. Note:: ``spack test`` requires version `0.16.0 <https://github.com/spack/spack/releases/tag/v0.16.0>`_ or higher in order to use this feature.
 
-buildtest can run tests using ``spack test run`` that can be used for testing installed specs with
-tests provided by spack. In order to run tests, you need to declare the ``test`` section
-which is of ``type: object`` in JSON and ``run`` is a required property. The ``run`` section maps to ``spack test run``
+buildtest can run tests via ``spack test run`` that can be used for testing installed specs which comes with builtin tests by the spack framework.
+In order to use this feature, you need to declare the ``test`` section
+which is ``type: object`` in JSON and ``run`` is a required property. The ``run`` section maps to ``spack test run``
 that is responsible for running tests for a list of specs that are specified using the ``specs`` property.
 
 Upon running the tests, we can retrieve results using ``spack test results`` which is configured using the ``results``
-property. The **results** property expects one to specify the ``specs`` or ``suite`` or both in order to retrieve results.
+property. The **results** property can query test results one of the following ways:
 
-The ``suite`` property is used to retrieve test results based on suite name, whereas ``specs`` property can be used to retrieve based
-on spec format. Both properties are a list of string types.
+1. Spec Format: ``spack test results -- <spec>``
+2. Suitename: ``spack test results <suitename>``
 
-In example below we will test `m4` package by running ``spack test run m4`` and report the results with log file.
+
+In example below, we will test **m4** package by running ``spack test run m4`` and specify the `-l` option (i.e `spack test results -l`)
+which will retrieve the test log.
 
 .. literalinclude:: ../../examples/spack/spack_test.yml
     :language: yaml
-    :emphasize-lines: 9-14
+    :emphasize-lines: 9-13
 
-If we look at the generated test, buildtest will install m4 followed by running the test. The **spack test run --alias**
-option is used to reference name of suitename which can be used to reference suitename when using ``spack test results``
-to search for test results. **buildtest will create the suite name based on name of test.** If `--alias` is not specified,
-spack will generate a random text for suitename which you won't know at time of writing test that is required by
-``spack test results`` to fetch the results.
+The **spack test run --alias** option is used to query results by suitename which can be used by ``spack test results`` command.
+**buildtest will create a unique suite name for every run so you don't have to remember the suite name when writing the buildspec.**
 
 .. dropdown:: ``buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml``
 
     .. program-output:: cat buildtest_tutorial_examples/spack/build/spack_test.txt
 
+Take note of the generated test and the suite-name that is generated by buildtest.
+
 .. dropdown:: ``buildtest inspect query -o -t spack_test_m4``
 
     .. program-output:: cat buildtest_tutorial_examples/spack/inspect/spack_test.txt
 
 
 We can search for test results using the spec format instead of suite name. In the ``results`` property we can
-use ``specs`` field instead of ``suite`` property to specify a list of spec names to run. In spack, you can retrieve
+use ``specs`` field to specify a list of spec names to run. In spack, you can retrieve
 the results using ``spack test results -- <spec>``, note that double dash ``--`` is in front of spec name. We can
 pass options to ``spack test results`` using the **option** property which is available for ``results`` and
 ``run`` property. Currently, spack will write test results in ``$HOME/.spack/tests`` and we can use ``spack test remove``


Test patch:
diff --git a/buildtest/builders/spack.py b/buildtest/builders/spack.py
index 6d86b6f33..7561a412f 100644
--- a/buildtest/builders/spack.py
+++ b/buildtest/builders/spack.py
@@ -4,6 +4,7 @@
 """
 
 import os
+import uuid
 
 from buildtest.builders.base import BuilderBase
 from buildtest.exceptions import BuildTestError
@@ -147,8 +148,8 @@ def _spack_test(self, spack_configuration):
             spack_test_cmd.append(spack_configuration["test"]["run"]["option"])
 
         run_specs = spack_configuration["test"]["run"]["specs"]
-
-        spack_test_cmd.append(f"--alias {self.name}")
+        suite_name = str(uuid.uuid4())
+        spack_test_cmd.append(f"--alias {suite_name}")
 
         for spec in run_specs:
             spack_test_cmd.append(spec)
@@ -156,16 +157,13 @@ def _spack_test(self, spack_configuration):
         lines.append(" ".join(spack_test_cmd))
 
         opts = spack_configuration["test"]["results"].get("option") or ""
-        # fetch results using 'spack test results <suite>'
-        if spack_configuration["test"]["results"].get("suite"):
-            for suite in spack_configuration["test"]["results"]["suite"]:
-                lines.append(f"spack test results {opts} {suite}")
 
         # fetch results using 'spack test results -- <spec>'
         if spack_configuration["test"]["results"].get("specs"):
             for spec in spack_configuration["test"]["results"]["specs"]:
                 lines.append(f"spack test results {opts} -- {spec}")
-
+        else:
+            lines.append(f"spack test results {opts} {suite_name}")
         return lines
 
     def _resolve_spack_root(self, path, verify_spack=True):
diff --git a/buildtest/schemas/examples/spack.schema.json/invalid/examples.yml b/buildtest/schemas/examples/spack.schema.json/invalid/examples.yml
index 2bc0df56d..6d4454de4 100644
--- a/buildtest/schemas/examples/spack.schema.json/invalid/examples.yml
+++ b/buildtest/schemas/examples/spack.schema.json/invalid/examples.yml
@@ -57,29 +57,6 @@ buildspecs:
         run:
           specs: ['m4', 'zlib']
 
-  spack_test_results_testing_required_oneOf:
-    type: spack
-    executor: generic.local.sh
-    description: "The results property expects one to specify specs, suite, or both"
-    tags: [spack]
-    pre_cmds: |
-      cd /tmp
-      git clone https://github.com/spack/spack
-    spack:
-      root: /tmp/spack
-      verify_spack: false
-      install:
-        specs: ['m4', 'zlib']
-      test:
-        remove_tests: true
-        run:
-          specs: ['m4', 'zlib']
-        results:
-          option: '-f'
-    post_cmds: |
-      spack find
-      rm -rf $SPACK_ROOT
-
   spack_test_run_invalid_spec:
     type: spack
     executor: generic.local.sh
@@ -99,9 +76,6 @@ buildspecs:
           specs: ['m4', 1]
         results:
           option: '-f'
-          suite: ['zlib']
-          specs: ['m4']
-
     post_cmds: |
       spack find
       rm -rf $SPACK_ROOT
diff --git a/buildtest/schemas/examples/spack.schema.json/valid/examples.yml b/buildtest/schemas/examples/spack.schema.json/valid/examples.yml
index d33e94fc4..591042efb 100644
--- a/buildtest/schemas/examples/spack.schema.json/valid/examples.yml
+++ b/buildtest/schemas/examples/spack.schema.json/valid/examples.yml
@@ -143,29 +143,6 @@ buildspecs:
         specs:
           - 'm4'
 
-
-  spack_test_results_specs_suites:
-    type: spack
-    executor: generic.local.sh
-    description: "Report results using suitename and spec format in spack test results"
-    tags: [spack]
-    pre_cmds: |
-      cd /tmp
-      git clone https://github.com/spack/spack
-    spack:
-      root: /tmp/spack
-      verify_spack: false
-      install:
-        specs: ['m4', 'zlib']
-      test:
-        remove_tests: true
-        run:
-          specs: ['m4', 'zlib']
-        results:
-          option: '-f'
-          suite: ['zlib']
-          specs: ['m4']
-
     post_cmds: |
       spack find
       rm -rf $SPACK_ROOT
diff --git a/buildtest/schemas/spack.schema.json b/buildtest/schemas/spack.schema.json
index 42ec6495d..2ab7b1fab 100644
--- a/buildtest/schemas/spack.schema.json
+++ b/buildtest/schemas/spack.schema.json
@@ -210,20 +210,11 @@
           "type": "object",
           "description": "View test results via ``spack test results`` after running tests via ``spack test run``. Results can be viewed using suitename or installed specs or both.",
           "additionalProperties": false,
-          "anyOf": [
-                    {"required": ["specs"] },
-                    {"required": ["suite"] },
-                    {"required": ["specs","suite"] }
-          ],
           "properties": {
             "option": {
               "type": "string",
               "description": "Pass options to ``spack test results``"
             },
-            "suite": {
-                  "$ref": "definitions.schema.json#/definitions/list_of_strings",
-                  "description": "Report results by  suite name by running ``spack test results <suite>``."
-            },
             "specs": {
                   "$ref": "definitions.schema.json#/definitions/list_of_strings",
                   "description": "Report result by spec name by running ``spack test run -- <specs>``."
diff --git a/docs/buildtest_tutorial_examples/spack/build/spack_test.txt b/docs/buildtest_tutorial_examples/spack/build/spack_test.txt
index eeb2a29a7..c4ef20ea2 100644
--- a/docs/buildtest_tutorial_examples/spack/build/spack_test.txt
+++ b/docs/buildtest_tutorial_examples/spack/build/spack_test.txt
@@ -1,22 +1,22 @@
 $ buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml 
-╭───────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────╮                                                                                                                   
-│                                                                                                                      │                                                                                                                   
-│ User:               spack                                                                                            │                                                                                                                   
-│ Hostname:           f215b09d0d2f                                                                                     │                                                                                                                   
-│ Platform:           Linux                                                                                            │                                                                                                                   
-│ Current Time:       2023/02/06 21:15:51                                                                              │                                                                                                                   
-│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                              │                                                                                                                   
-│ buildtest version:  1.1                                                                                              │                                                                                                                   
-│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                          │                                                                                                                   
-│ python version:     3.8.6                                                                                            │                                                                                                                   
-│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                     │                                                                                                                   
-│ Test Directory:     /home/spack/runs                                                                                 │                                                                                                                   
-│ Report File:        /home/spack/buildtest/var/report.json                                                            │                                                                                                                   
-│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml │                                                                                                                   
-│                                                                                                                      │                                                                                                                   
-╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                                   
-─────────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────
-                 Discovered buildspecs                 
+╭───────────────────────────────────── buildtest summary ─────────────────────────────────────╮
+│                                                                                             │
+│ User:               spack                                                                   │
+│ Hostname:           94e3cbc65331                                                            │
+│ Platform:           Linux                                                                   │
+│ Current Time:       2023/04/25 13:45:42                                                     │
+│ buildtest path:     /home/spack/buildtest/bin/buildtest                                     │
+│ buildtest version:  1.3                                                                     │
+│ python path:        /home/spack/pyenv/buildtest/bin/python3                                 │
+│ python version:     3.8.6                                                                   │
+│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml            │
+│ Test Directory:     /home/spack/runs                                                        │
+│ Report File:        /home/spack/buildtest/var/report.json                                   │
+│ Command:            /home/spack/buildtest/bin/buildtest bd -b examples/spack/spack_test.yml │
+│                                                                                             │
+╰─────────────────────────────────────────────────────────────────────────────────────────────╯
+───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
+                 Discovered buildspecs
 ╔═════════════════════════════════════════════════════╗
 ║ buildspec                                           ║
 ╟─────────────────────────────────────────────────────╢
@@ -27,37 +27,37 @@ $ buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml
 Total Discovered Buildspecs:  1
 Total Excluded Buildspecs:  0
 Detected Buildspecs after exclusion:  1
-─────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────
+────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
 Valid Buildspecs: 1
 Invalid Buildspecs: 0
 /home/spack/buildtest/examples/spack/spack_test.yml: VALID
 Total builder objects created: 1
-                                                                                  Builders by type=spack                                                                                   
+                                                                                  Builders by type=spack
 ┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
 ┃ builder                ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                      ┃ buildspecs                                          ┃
 ┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
-│ spack_test_m4/460eadb0 │ spack │ generic.local.bash │ None     │ None  │ None  │ Run spack test for m4 package and report results │ /home/spack/buildtest/examples/spack/spack_test.yml │
+│ spack_test_m4/63314812 │ spack │ generic.local.bash │ None     │ None  │ None  │ Run spack test for m4 package and report results │ /home/spack/buildtest/examples/spack/spack_test.yml │
 └────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘
-────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
-spack_test_m4/460eadb0: Creating test directory: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0
-spack_test_m4/460eadb0: Creating the stage directory: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/stage
-spack_test_m4/460eadb0: Writing build script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4_build.sh
-────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
+──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
+spack_test_m4/63314812: Creating test directory: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812
+spack_test_m4/63314812: Creating the stage directory: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/stage
+spack_test_m4/63314812: Writing build script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4_build.sh
+──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
 Spawning 4 processes for processing builders
-─────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────
-spack_test_m4/460eadb0 does not have any dependencies adding test to queue
-spack_test_m4/460eadb0: Current Working Directory : /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/stage
-spack_test_m4/460eadb0: Running Test via command: bash --norc --noprofile -eo pipefail spack_test_m4_build.sh
-spack_test_m4/460eadb0: Test completed in 2.1575 seconds
-spack_test_m4/460eadb0: Test completed with returncode: 0
-spack_test_m4/460eadb0: Writing output file -  /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4.out
-spack_test_m4/460eadb0: Writing error file - /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4.err
-In this iteration we are going to run the following tests: [spack_test_m4/460eadb0]
-                                                    Test Summary                                                     
+───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
+spack_test_m4/63314812 does not have any dependencies adding test to queue
+In this iteration we are going to run the following tests: [spack_test_m4/63314812]
+spack_test_m4/63314812: Current Working Directory : /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/stage
+spack_test_m4/63314812: Running Test via command: bash --norc --noprofile -eo pipefail spack_test_m4_build.sh
+spack_test_m4/63314812: Test completed in 1.39538 seconds
+spack_test_m4/63314812: Test completed with returncode: 0
+spack_test_m4/63314812: Writing output file -  /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4.out
+spack_test_m4/63314812: Writing error file - /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4.err
+                                                    Test Summary
 ┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
 ┃ builder                ┃ executor           ┃ status ┃ checks (ReturnCode, Regex, Runtime) ┃ returncode ┃ runtime ┃
 ┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
-│ spack_test_m4/460eadb0 │ generic.local.bash │ PASS   │ N/A N/A N/A                         │ 0          │ 2.1575  │
+│ spack_test_m4/63314812 │ generic.local.bash │ PASS   │ None None None                      │ 0          │ 1.39538 │
 └────────────────────────┴────────────────────┴────────┴─────────────────────────────────────┴────────────┴─────────┘
 
 
@@ -67,4 +67,4 @@ Failed Tests: 0/1 Percentage: 0.000%
 
 
 Adding 1 test results to /home/spack/buildtest/var/report.json
-Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_bpf4x5c3.log
+Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_q357l26y.log
diff --git a/docs/buildtest_tutorial_examples/spack/inspect/spack_test.txt b/docs/buildtest_tutorial_examples/spack/inspect/spack_test.txt
index c5dde4769..163da2425 100644
--- a/docs/buildtest_tutorial_examples/spack/inspect/spack_test.txt
+++ b/docs/buildtest_tutorial_examples/spack/inspect/spack_test.txt
@@ -1,41 +1,41 @@
 $ buildtest inspect query -o -t spack_test_m4 
-─────────────────────────────────────────────────────────────────────────────────────────── spack_test_m4/460eadb0-6966-41db-bc77-1a00af9fe500 ────────────────────────────────────────────────────────────────────────────────────────────
+────────────────────────────────────────────────────────────────────────────────────────── spack_test_m4/63314812-fcd6-4aeb-a51e-c6cb3924d171 ──────────────────────────────────────────────────────────────────────────────────────────
 Executor: generic.local.bash
 Description: Run spack test for m4 package and report results
 State: PASS
 Returncode: 0
-Runtime: 2.1575 sec
-Starttime: 2023/02/06 21:15:51
-Endtime: 2023/02/06 21:15:53
+Runtime: 1.39538 sec
+Starttime: 2023/04/25 13:45:42
+Endtime: 2023/04/25 13:45:44
 Command: bash --norc --noprofile -eo pipefail spack_test_m4_build.sh
-Test Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4.sh
-Build Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4_build.sh
-Output File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4.out
-Error File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4.err
-Log File: /home/spack/buildtest/var/logs/buildtest_bpf4x5c3.log
-────────────────────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4.out ───────────────────────────────────────────────────────────────────
-==> Spack test spack_test_m4                                                                                                                                                                                                               
-==> Testing package m4-1.4.18-mkc3u4x                                                                                                                                                                                                      
-==> Results for test suite 'spack_test_m4':                                                                                                                                                                                                
-==>   m4-1.4.18-mkc3u4x PASSED                                                                                                                                                                                                             
-==> Testing package m4-1.4.18-mkc3u4x                                                                                                                                                                                                      
-==> [2023-02-06-21:15:53.383170] test: ensuring m4 version is 1.4.18                                                                                                                                                                       
-==> [2023-02-06-21:15:53.384286] '/home/spack/spack-workspace/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/m4-1.4.18-mkc3u4x2p2wie6jfhuku7g5rkovcrxps/bin/m4' '--version'                                                                   
-m4 (GNU M4) 1.4.18                                                                                                                                                                                                                         
-Copyright (C) 2016 Free Software Foundation, Inc.                                                                                                                                                                                          
-License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.                                                                                                                                                             
-This is free software: you are free to change and redistribute it.                                                                                                                                                                         
-There is NO WARRANTY, to the extent permitted by law.                                                                                                                                                                                      
-Written by Rene' Seindal.                                                                                                                                                                                                                  
-PASSED                                                                                                                                                                                                                                     
-==> [2023-02-06-21:15:53.392419] test: ensuring m4 example succeeds                                                                                                                                                                        
-==> [2023-02-06-21:15:53.392722] '/home/spack/spack-workspace/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/m4-1.4.18-mkc3u4x2p2wie6jfhuku7g5rkovcrxps/bin/m4' '/home/spack/.spack/test/5mpugnenojhehp5km573ucervxz46odi/m4-1.4.18-mkc3u4x/da
-// macro is defined                                                                                                                                                                                                                        
-Hello, World!                                                                                                                                                                                                                              
-PASSED                                                                                                                                                                                                                                     
-                                                                                                                                                                                                                                           
-──────────────────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/460eadb0/spack_test_m4.sh ────────────────────────────────────────────────────────────────────
-#!/bin/bash                                                                                                                                                                                                                                
-source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                          
-spack test run --alias spack_test_m4 m4                                                                                                                                                                                                    
-spack test results -l spack_test_m4                                                                                                                                                                                                        
+Test Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4.sh
+Build Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4_build.sh
+Output File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4.out
+Error File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4.err
+Log File: /home/spack/buildtest/var/logs/buildtest_q357l26y.log
+───────────────────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4.out ─────────────────────────────────────────────────────────────────
+==> Spack test 2f2840ce-3f38-4c2d-afde-9e1065dfbe86
+==> Testing package m4-1.4.18-mkc3u4x
+==> Results for test suite '2f2840ce-3f38-4c2d-afde-9e1065dfbe86':
+==>   m4-1.4.18-mkc3u4x PASSED
+==> Testing package m4-1.4.18-mkc3u4x
+==> [2023-04-25-13:45:43.698955] test: ensuring m4 version is 1.4.18
+==> [2023-04-25-13:45:43.699920] '/home/spack/spack-workspace/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/m4-1.4.18-mkc3u4x2p2wie6jfhuku7g5rkovcrxps/bin/m4' '--version'
+m4 (GNU M4) 1.4.18
+Copyright (C) 2016 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.
+Written by Rene' Seindal.
+PASSED
+==> [2023-04-25-13:45:43.704828] test: ensuring m4 example succeeds
+==> [2023-04-25-13:45:43.705080] '/home/spack/spack-workspace/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/m4-1.4.18-mkc3u4x2p2wie6jfhuku7g5rkovcrxps/bin/m4' '/home/spack/.spack/test/j22esx4avlcbuqpourbkp7wzftm7ilj3/m4-1.4.18-mkc3u4x
+// macro is defined
+Hello, World!
+PASSED
+
+────────────────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/63314812/spack_test_m4.sh ───────────────────────────────────────────────────────────────────
+#!/bin/bash
+source /home/spack/spack/share/spack/setup-env.sh
+spack test run --alias 2f2840ce-3f38-4c2d-afde-9e1065dfbe86 m4
+spack test results -l 2f2840ce-3f38-4c2d-afde-9e1065dfbe86
\ No newline at end of file
diff --git a/examples/spack/spack_test.yml b/examples/spack/spack_test.yml
index 71d7f70b3..1466bf4b6 100644
--- a/examples/spack/spack_test.yml
+++ b/examples/spack/spack_test.yml
@@ -10,5 +10,4 @@ buildspecs:
         run:
           specs: ['m4']
         results:
-          suite: ['spack_test_m4']
           option: "-l"
\ No newline at end of file