Repository: EveryVoiceTTS/EveryVoice
Pull request #648: Dev.ej/regress rebooted
Author: login: joanise
Created: 2025-05-02T18:51:18Z | Merged: 2025-05-30T17:41:15Z

<!-- PR template: please provide enough information to guide your reviewers.
Please read Contributing.md before submitting a PR.  -->

### PR Goal? <!-- Explain the main objective of this PR. -->

The overall goal is to improve the regression test suite in various ways.

1) improve the coverage reports
2) use playwright to exercise the demo in testing
3) factor out how to submit regression on different clusters
4) add a mix-corpus test case in the regression suite

I know it would be better to splice this is 2 or 3 PRs, but I didn't leave myself enough time to do that before Jordan arrives and I'm hoping he'll work on top of this branch, so I'm submitting it now like this.

### Feedback sought? <!-- What should reviewers focus on in particular? -->

Flag egregious issues only. This is still work in progress anyway, but I think it's in a state that's already worth merging, and we'll keep improving it in future PRs.

### Priority? <!-- How soon would you like this PR reviewed, does it block other work? -->

high

### Tests added? <!-- Make sure your PR includes automated tests for your changes. -->

it's all just testing

### How to test? <!-- Explain how reviewers should test this PR. -->

run the regression suites on a cluster

### Confidence? <!-- How confident are you that these changes are ready to merge? -->

high that it's ok to merge now, but I know we'll keep improving on this.

### Version change? <!-- Do you think this PR should trigger a Major (Breaking Change)/Minor (New Feature)/patch (refactor/bug fix) version change? -->

no

### Related PRs? <!-- List any submodule PRs required for this PR to make sense. -->

no

<!-- Add any other relevant information here -->



Diff:
diff --git a/everyvoice/demo/app.py b/everyvoice/demo/app.py
index b2c3a7f7..bdc2664f 100644
--- a/everyvoice/demo/app.py
+++ b/everyvoice/demo/app.py
@@ -314,7 +314,7 @@ def create_demo_app(
                     # When the only output option is wav, don't show the File Output box
                     outputs = [out_audio]
                 else:
-                    out_file = gr.File(label="File Output")
+                    out_file = gr.File(label="File Output", elem_id="file_output")
                     outputs = [out_audio, out_file]
         # Only include the style reference input if the model supports it
         if model.config.model.use_global_style_token_module:
diff --git a/everyvoice/tests/regression/.gitignore b/everyvoice/tests/regression/.gitignore
index 0e71ce9b..d1921a87 100644
--- a/everyvoice/tests/regression/.gitignore
+++ b/everyvoice/tests/regression/.gitignore
@@ -1,3 +1,3 @@
 EV-regress.*
-EV-r-main.*
+EV*r-main.*
 regress-*
diff --git a/everyvoice/tests/regression/README.md b/everyvoice/tests/regression/README.md
index 5effd6a3..2c728d5d 100644
--- a/everyvoice/tests/regression/README.md
+++ b/everyvoice/tests/regression/README.md
@@ -2,11 +2,12 @@
 
 ## Preparing the regression training data:
 
-- Download LJ 1.1 from https://keithito.com/LJ-Speech-Dataset/
-- Download Sinhala TTS from https://openslr.org/30/
+- Download LJ 1.1 from https://keithito.com/LJ-Speech-Dataset/ into $HOME/sgile/data/LJSpeech-1.1
+- Download Sinhala TTS from https://openslr.org/30/ into $HOME/sgile/data/SinhalaTTS
 - Download High quality TTS data for four South African languages (af, st, tn,
-  xh) from https://openslr.org/32
-- See [`prep-datasets.sh`](prep-datasets.sh) to see where these datasets are expected to be found.
+  xh) from https://openslr.org/32 into $HOME/sgile/data/OpenSLR32-four-South-Afican-languages
+- See [`prep-datasets.sh`](prep-datasets.sh) to see exactly where these datasets
+  are expected to be found.
 - Run this to create the regression testing directory structure:
 
 ```sh
@@ -38,6 +39,6 @@ All the above can be accomplished by running `go.sh`.
 
 ## Cluster parameters
 
-The scripts hardcode NRC's default Slurm cluster parameters. Add `--partition=... --account=...`
-the the `sbatch` commands to override, or edit `go.sh` and `regression-test.sh` to use
-your partition and account settings to request nodes with GPUs available.
+For NRC clusters, use `sbatch go-<clustername>.sh` to get the appropriate Slurm
+parameters. To run this on a different cluster, copy one of the go-*.sh scripts
+and customize it with the right partition and account for you.
diff --git a/everyvoice/tests/regression/combine-coverage.sh b/everyvoice/tests/regression/combine-coverage.sh
index c3f47ce7..17818038 100755
--- a/everyvoice/tests/regression/combine-coverage.sh
+++ b/everyvoice/tests/regression/combine-coverage.sh
@@ -1,7 +1,12 @@
 #!/bin/bash
 
-find . -name .coverage\* | coverage combine --keep
-coverage report --include='*/everyvoice/*' | sed 's/.*EveryVoice\/everyvoice/everyvoice/' > coverage.txt
-coverage html --include='*/everyvoice/*'
-coverage xml --include='*/everyvoice/*'
-sed -i 's/"[^"]*EveryVoice.everyvoice/"everyvoice/g' coverage.xml
+find . -name .coverage.\* | xargs coverage combine --keep
+
+COVERAGE_DIR=$(pwd -P)
+EVERYVOICE_ROOT=$(python -c 'import everyvoice; print(everyvoice.__path__[0])')/..
+COVERAGE_OPTIONS=(--data-file="$COVERAGE_DIR/.coverage" --include='*/everyvoice/*')
+
+cd "$EVERYVOICE_ROOT" || exit 1
+coverage report "${COVERAGE_OPTIONS[@]}" > "$COVERAGE_DIR/coverage.txt"
+coverage html "${COVERAGE_OPTIONS[@]}" --directory="$COVERAGE_DIR/htmlcov"
+coverage xml "${COVERAGE_OPTIONS[@]}" -o "$COVERAGE_DIR/coverage.xml"
diff --git a/everyvoice/tests/regression/go-gpsc5.sh b/everyvoice/tests/regression/go-gpsc5.sh
new file mode 100644
index 00000000..45e1e849
--- /dev/null
+++ b/everyvoice/tests/regression/go-gpsc5.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+#SBATCH --job-name=EVr-main
+#SBATCH --partition=standard
+#SBATCH --account=nrc_ict
+#SBATCH --qos=low
+#SBATCH --time=720
+#SBATCH --ntasks=1
+#SBATCH --cpus-per-task=1
+#SBATCH --mem=8000M
+#SBATCH --output=./%x.o%j
+#SBATCH --error=./%x.e%j
+
+# Command to submit a GPU job:
+export SUBMIT_COMMAND="sbatch --qos=low --partition=gpu_v100 --account=nrc_ict__gpu_v100"
+bash go.sh
diff --git a/everyvoice/tests/regression/go-gpscc.sh b/everyvoice/tests/regression/go-gpscc.sh
new file mode 100644
index 00000000..a8799b6e
--- /dev/null
+++ b/everyvoice/tests/regression/go-gpscc.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+#SBATCH --job-name=EVr-main
+#SBATCH --partition=standard
+#SBATCH --account=nrc_ict
+#SBATCH --qos=low
+#SBATCH --time=720
+#SBATCH --ntasks=1
+#SBATCH --cpus-per-task=1
+#SBATCH --mem=8000M
+#SBATCH --output=./%x.o%j
+#SBATCH --error=./%x.e%j
+
+# Command to submit a GPU job:
+export SUBMIT_COMMAND="sbatch --qos=low --partition=gpu_a100 --account=nrc_ict__gpu_a100"
+bash go.sh
diff --git a/everyvoice/tests/regression/go-trixie.sh b/everyvoice/tests/regression/go-trixie.sh
new file mode 100644
index 00000000..6520d8f2
--- /dev/null
+++ b/everyvoice/tests/regression/go-trixie.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+#SBATCH --job-name=EVr-main
+#SBATCH --partition=TrixieMain
+#SBATCH --time=720
+#SBATCH --ntasks=1
+#SBATCH --cpus-per-task=1
+#SBATCH --mem=8000M
+#SBATCH --output=./%x.o%j
+#SBATCH --error=./%x.e%j
+
+# Command to submit a GPU job:
+export SUBMIT_COMMAND="sbatch --partition=TrixieMain"
+bash go.sh
diff --git a/everyvoice/tests/regression/go.sh b/everyvoice/tests/regression/go.sh
index f488e239..424e5d0e 100755
--- a/everyvoice/tests/regression/go.sh
+++ b/everyvoice/tests/regression/go.sh
@@ -1,16 +1,5 @@
 #!/bin/bash
 
-#SBATCH --job-name=EV-r-main
-#SBATCH --partition=standard
-#SBATCH --account=nrc_ict
-#SBATCH --qos=low
-#SBATCH --time=10080
-#SBATCH --ntasks=1
-#SBATCH --cpus-per-task=1
-#SBATCH --mem=8000M
-#SBATCH --output=./%x.o%j
-#SBATCH --error=./%x.e%j
-
 # Automated application of the instructions in README.md
 
 set -o errexit
@@ -18,36 +7,45 @@ set -o errexit
 TOP_LEVEL_DIR=$(mktemp --directory regress-$(date +'%Y%m%d')-XXX)
 cd "$TOP_LEVEL_DIR"
 
-if sbatch -h >& /dev/null; then
-    SUBMIT_COMMAND=sbatch
-else
+# Cluster-specific calling script should set SUBMIT_COMMAND appropriately
+# to request a partition where GPU jobs can run.
+# For non-cluster contexts, default to running subjobs sequentially with bash.
+if [[ ! $SUBMIT_COMMAND ]]; then
     SUBMIT_COMMAND=bash
 fi
 
+if ! python -c $'from playwright.sync_api import sync_playwright\nwith sync_playwright() as playwright: playwright.chromium.launch(headless=True)'; then
+    echo '
+ERROR: playwright headless chromium browser not found.
+Please install playwright by installing test dependencies with
+    pip install -e .[test]
+and then the headless chromium brownser with:
+    playwright install --only-shell chromium'
+    exit 1
+fi
+
+# Save some version info so we know what this was run on
+(
+    echo git describe: "$(git describe)"
+    echo git status:
+    git status
+) | tee version.txt
+
 ../prep-datasets.sh
+SUBMIT_OPTIONS=""
 for DIR in regress-*; do
     pushd "$DIR"
-    $SUBMIT_COMMAND ../../regression-test.sh
+    if [[ $SUBMIT_COMMAND =~ sbatch ]]; then
+        SHORT_DIR=EVr-${DIR#regress-}
+        SUBMIT_OPTIONS="--job-name $SHORT_DIR"
+    fi
+    $SUBMIT_COMMAND $SUBMIT_OPTIONS ../../regression-test.sh
     popd
 done
 
-coverage run -p -m everyvoice test
+coverage run -p -m everyvoice test 2>&1
 
 JOB_COUNT=$(find . -maxdepth 1 -name regress-\* | wc -l)
-while true; do
-    DONE_COUNT=$(find . -maxdepth 2 -name DONE | wc -l)
-    if (( DONE_COUNT + 2 >= JOB_COUNT )); then
-        break
-    fi
-    echo "$DONE_COUNT/$JOB_COUNT regression job(s) done. Still waiting."
-    date
-    sleep $(( 60 * 5 ))
-done
-
-echo "$DONE_COUNT regression jobs done. Calculating coverage now, but some jobs may still be running."
-../combine-coverage.sh
-cat coverage.txt
-
 while true; do
     DONE_COUNT=$(find . -maxdepth 2 -name DONE | wc -l)
     if (( DONE_COUNT >= JOB_COUNT )); then
@@ -55,10 +53,23 @@ while true; do
     fi
     echo "$DONE_COUNT/$JOB_COUNT regression job(s) done. Still waiting."
     date
-    sleep $(( 60 * 5 ))
+    # Update coverage reports every polling period in case we want to follow it.
+    rm -f .coverage
+    ../combine-coverage.sh > /dev/null
+    sleep 60
 done
 
 echo "All $DONE_COUNT regression jobs done. Calculating final coverage."
-rm .coverage
+# Calculate coverage
+rm -f .coverage
 ../combine-coverage.sh
 cat coverage.txt
+
+# Calculate some helpful coverage diffs: from origin/main and from the last published version.
+(
+    REGRESS_DIR=$(pwd)
+    cd ../../../..
+    diff-cover --compare-branch origin/main "$REGRESS_DIR"/coverage.xml --html-report "$REGRESS_DIR"/coverage-diff-main.html
+    LAST_VERSION=$(git describe | sed 's/-.*//')
+    diff-cover --compare-branch "$LAST_VERSION" "$REGRESS_DIR"/coverage.xml --html-report "$REGRESS_DIR"/coverage-diff-v0.2.0a1.html
+) | tee coverage-diff.txt
diff --git a/everyvoice/tests/regression/prep-datasets.sh b/everyvoice/tests/regression/prep-datasets.sh
index 4b6ff9ea..5c74f8b3 100755
--- a/everyvoice/tests/regression/prep-datasets.sh
+++ b/everyvoice/tests/regression/prep-datasets.sh
@@ -35,6 +35,9 @@ Boo!
 ==EOF==
     echo spec > "$dir"/test2.txt
 done
+cp "$EVERYVOICE_REGRESS_ROOT"/run-demo-app-lj-full.sh regress-lj-full/run-demo-app.sh
+cp "$EVERYVOICE_REGRESS_ROOT"/test-demo-app-lj-full.py regress-lj-full/test-demo-app.py
+cp "$EVERYVOICE_REGRESS_ROOT"/wait-for-demo-app.py "$dir"/wait-for-demo-app.py
 
 SinhalaTTS=$SGILE_DATASET_ROOT/SinhalaTTS
 dir=regress-si
@@ -67,3 +70,16 @@ uqeqesho
 iimpumlo
 ==EOF==
 echo isiXhosa > "$dir"/test2.txt
+
+dir=regress-mix
+mkdir $dir
+cp "$EVERYVOICE_REGRESS_ROOT"/wizard-resume-mix "$dir"/wizard-resume
+cat <<'==EOF==' > "$dir"/test.txt
+This is a test.
+සිංහල අක්ෂර මාලාව
+ukukrwentshwa isiXhosa
+==EOF==
+echo test > "$dir"/test2.txt
+cp "$EVERYVOICE_REGRESS_ROOT"/run-demo-app-mix.sh "$dir"/run-demo-app.sh
+cp "$EVERYVOICE_REGRESS_ROOT"/test-demo-app-mix.py "$dir"/test-demo-app.py
+cp "$EVERYVOICE_REGRESS_ROOT"/wait-for-demo-app.py "$dir"/wait-for-demo-app.py
diff --git a/everyvoice/tests/regression/regression-test.sh b/everyvoice/tests/regression/regression-test.sh
index 30f69423..fa101c2c 100755
--- a/everyvoice/tests/regression/regression-test.sh
+++ b/everyvoice/tests/regression/regression-test.sh
@@ -1,9 +1,6 @@
 #!/bin/bash
 
 #SBATCH --job-name=EV-regress
-#SBATCH --partition=gpu_a100
-#SBATCH --account=nrc_ict__gpu_a100
-#SBATCH --qos=low
 #SBATCH --time=180
 #SBATCH --ntasks=1
 #SBATCH --cpus-per-task=4
@@ -89,12 +86,19 @@ r "coverage run -p -m everyvoice synthesize from-spec \
 #r "coverage run -p -m dfaligner extract-alignments config/everyvoice-aligner.yaml --model '$ALIGNER'"
 
 
-# Spin up the demo
-# everyvoice demo $FS2 $VOCODER &
-# TODO: use playwright to synthesize something using the demo
+# Spin up the demo and exercise it with Playwright in headless mode
+if [[ -f ../run-demo-app.sh && -f ../test-demo-app.py ]]; then
+    bash ../run-demo-app.sh &
+    DEMO_PID=$!
+    sleep 10
+    python ../wait-for-demo-app.py
+    coverage run -p ../test-demo-app.py
+    kill $DEMO_PID
+    sleep 5
+fi
 
 
-# TODO: use coverage analysis to flag the next priority things to add here
+# TODO: use coverage analysis to find the next priority things to add here
 
 
 echo "Done at $(date)"
diff --git a/everyvoice/tests/regression/run-demo-app-lj-full.sh b/everyvoice/tests/regression/run-demo-app-lj-full.sh
new file mode 100644
index 00000000..cbb58cb2
--- /dev/null
+++ b/everyvoice/tests/regression/run-demo-app-lj-full.sh
@@ -0,0 +1 @@
+coverage run -p -m everyvoice demo logs_and_checkpoints/FeaturePredictionExperiment/base/checkpoints/last.ckpt logs_and_checkpoints/VocoderExperiment/base/checkpoints/last.ckpt --denylist <(echo banned) --output-format wav
diff --git a/everyvoice/tests/regression/run-demo-app-mix.sh b/everyvoice/tests/regression/run-demo-app-mix.sh
new file mode 100644
index 00000000..de6fd73e
--- /dev/null
+++ b/everyvoice/tests/regression/run-demo-app-mix.sh
@@ -0,0 +1 @@
+coverage run -p -m everyvoice demo logs_and_checkpoints/FeaturePredictionExperiment/base/checkpoints/last.ckpt logs_and_checkpoints/VocoderExperiment/base/checkpoints/last.ckpt
diff --git a/everyvoice/tests/regression/test-demo-app-lj-full.py b/everyvoice/tests/regression/test-demo-app-lj-full.py
new file mode 100644
index 00000000..dc4258b8
--- /dev/null
+++ b/everyvoice/tests/regression/test-demo-app-lj-full.py
@@ -0,0 +1,53 @@
+"""
+Test the Demo App with the regress-lj-full demo model with a denylist.
+To run this test:
+ - Run the regression test suite to get the regress-lj-full model.
+ - cd regression/regress-<date>-<suffix>/regress-lj-full/regress
+ - everyvoice demo logs_and_checkpoints/FeaturePredictionExperiment/base/checkpoints/last.ckpt logs_and_checkpoints/VocoderExperiment/base/checkpoints/last.ckpt --denylist <(echo test) --output-format wav
+ - python test_demo_app_mix.py
+"""
+
+from unittest import TestCase, main
+
+from playwright.sync_api import expect, sync_playwright
+
+
+class TestDemoAppErros(TestCase):
+    def test_rundemo(self) -> None:
+        with sync_playwright() as playwright:
+            browser = playwright.chromium.launch(headless=True)
+            context = browser.new_context()
+            page = context.new_page()
+            page.goto("http://127.0.0.1:7860/")
+            synthesize_button = page.get_by_role("button", name="Synthesize")
+            synthesize_button.click()
+            expect(
+                page.get_by_text("Text for synthesis was not provided.")
+            ).to_be_visible()
+            page.get_by_test_id("toast-close").click()
+            expect(page.get_by_label("Play", exact=True)).to_have_count(0)
+
+            input_textbox = page.get_by_label("Input Text")
+            input_textbox.click()
+            input_textbox.fill("test denylist")
+            synthesize_button.click()
+            expect(
+                page.get_by_text(
+                    "Oops, the word test denylist is not allowed to be synthesized by this model."
+                )
+            ).to_be_visible()
+            page.get_by_test_id("toast-close").click()
+            expect(page.get_by_label("Play", exact=True)).to_have_count(0)
+
+            input_textbox.click()
+            input_textbox.fill("This is allowed.")
+            synthesize_button.click()
+            expect(page.get_by_label("Play", exact=True)).to_be_visible()
+
+            # ---------------------
+            context.close()
+            browser.close()
+
+
+if __name__ == "__main__":
+    main()
diff --git a/everyvoice/tests/regression/test-demo-app-mix.py b/everyvoice/tests/regression/test-demo-app-mix.py
new file mode 100644
index 00000000..3b82dd9f
--- /dev/null
+++ b/everyvoice/tests/regression/test-demo-app-mix.py
@@ -0,0 +1,88 @@
+"""
+Test the Demo App with the regress-mix demo model.
+To run this test:
+ - Run the regression test suite to get the regress-mix model.
+ - cd regression/regress-<date>-<suffix>/regress-mix/regress
+ - everyvoice demo logs_and_checkpoints/FeaturePredictionExperiment/base/checkpoints/last.ckpt logs_and_checkpoints/VocoderExperiment/base/checkpoints/last.ckpt
+ - python test_demo_app_mix.py
+"""
+
+from unittest import TestCase, main
+
+from playwright.sync_api import sync_playwright
+
+
+class TestDemoAppMix(TestCase):
+    def test_rundemo(self) -> None:
+        with sync_playwright() as playwright:
+            browser = playwright.chromium.launch(headless=True)
+            context = browser.new_context()
+            page = context.new_page()
+            page.goto("http://127.0.0.1:7860/")
+            input_textbox = page.get_by_label("Input Text")
+            input_textbox.click()
+            input_textbox.fill("This is a test.")
+            synthesize_button = page.get_by_role("button", name="Synthesize")
+            synthesize_button.click()
+            page.get_by_label("Play", exact=True).click()
+            with page.expect_download() as download_info:
+                page.get_by_label("Download").click()
+            download = download_info.value
+            self.assertTrue(download.suggested_filename.endswith(".wav"))
+            page.get_by_label("Output Format").click()
+            page.get_by_label("spec").click()
+            synthesize_button.click()
+            page.get_by_label("Play", exact=True).click()
+            with page.expect_download() as download1_info:
+                page.locator("#file_output").get_by_role("link").click()
+            download = download1_info.value
+            self.assertTrue(download.suggested_filename.endswith(".pt"))
+            page.get_by_label("Output Format").click()
+            page.get_by_label("textgrid").click()
+            synthesize_button.click()
+            with page.expect_download() as download2_info:
+                page.locator("#file_output").get_by_role("link").click()
+            download = download2_info.value
+            self.assertTrue(download.suggested_filename.endswith(".TextGrid"))
+            page.get_by_label("Output Format").click()
+            page.get_by_label("readalong-xml").click()
+            synthesize_button.click()
+            with page.expect_download() as download3_info:
+                page.locator("#file_output").get_by_role("link").click()
+            download = download3_info.value
+            self.assertTrue(download.suggested_filename.endswith(".readalong"))
+            page.get_by_label("Output Format").click()
+            page.get_by_label("readalong-html").click()
+            synthesize_button.click()
+            with page.expect_download() as download4_info:
+                page.locator("#file_output").get_by_role("link").click()
+            download = download4_info.value
+            self.assertTrue(download.suggested_filename.endswith(".html"))
+            page.get_by_label("Language").click()
+            page.get_by_label("und").click()
+            page.get_by_label("Speaker", exact=True).click()
+            page.get_by_label("isixhosa_speaker").click()
+            page.get_by_label("Output Format").click()
+            page.get_by_label("wav").click()
+            input_textbox.click()
+            input_textbox.fill("enye intlanzi imke negwegwe lam.")
+            synthesize_button.click()
+            page.get_by_label("Play", exact=True).click()
+            page.get_by_label("Speaker", exact=True).click()
+            page.get_by_label("sinhalese_speaker").click()
+            input_textbox.click()
+            input_textbox.fill("අටවිසියක් සෙනෙවියෝ හාත්පස රැකවල්ලා")
+            synthesize_button.click()
+            page.get_by_label("Play", exact=True).click()
+            input_textbox.click()
+            input_textbox.fill("This is a test.")
+            synthesize_button.click()
+            page.get_by_label("Play", exact=True).click()
+
+            # ---------------------
+            context.close()
+            browser.close()
+
+
+if __name__ == "__main__":
+    main()
diff --git a/everyvoice/tests/regression/wait-for-demo-app.py b/everyvoice/tests/regression/wait-for-demo-app.py
new file mode 100644
index 00000000..ace16277
--- /dev/null
+++ b/everyvoice/tests/regression/wait-for-demo-app.py
@@ -0,0 +1,14 @@
+from playwright.sync_api import sync_playwright
+
+with sync_playwright() as playwright:
+    browser = playwright.chromium.launch(headless=True)
+    context = browser.new_context()
+    page = context.new_page()
+    for _ in range(60):
+        try:
+            page.goto("http://127.0.0.1:7860/", timeout=5000)
+            print("Page loaded")
+            break
+        except Exception as e:
+            print("Error loading page:", e)
+            page.wait_for_timeout(1000)
diff --git a/everyvoice/tests/regression/wizard-resume-mix b/everyvoice/tests/regression/wizard-resume-mix
new file mode 100644
index 00000000..2ae7e8be
--- /dev/null
+++ b/everyvoice/tests/regression/wizard-resume-mix
@@ -0,0 +1,126 @@
+- - EveryVoice Wizard
+  - 0.2.0a1
+- - Root
+  - null
+- - Name Step
+  - regress
+- - Contact Name Step
+  - Regression Tester
+- - Contact Email Step
+  - Eric.Joanis@nrc-cnrc.gc.ca
+- - Output Path Step
+  - .
+- - Filelist Step
+  - ../regress-lj-600/metadata.csv
+- - Dataset Permission Step
+  - Yes, I do have permission to use this data.
+- - Filelist Format Step
+  - psv
+- - Filelist Has Header Line Step
+  - 'no'
+- - Basename Header Step
+  - 0
+- - Text Header Step
+  - 1
+- - Filelist Text Representation Step
+  - characters
+- - Text Processing Step
+  - - 0
+    - 1
+- - Data Has Speaker Step
+  - 'no'
+- - Know Speaker Step
+  - 'yes'
+- - Add Speaker Step
+  - english_speaker
+- - Data Has Language Step
+  - 'no'
+- - Select Language Step
+  - '[eng]: English'
+- - Wavs Dir Step
+  - ../regress-lj-600/wavs
+- - Validate Wavs Step
+  - OK
+- - Symbol-Set Step
+  - true
+- - SoX Effects Step
+  - []
+- - Dataset Name Step
+  - lj600
+- - More Datasets Step
+  - 'yes'
+- - Filelist Step
+  - ../regress-si/si_lk.lines.txt
+- - Dataset Permission Step
+  - Yes, I do have permission to use this data.
+- - Filelist Format Step
+  - festival
+- - Filelist Text Representation Step
+  - characters
+- - Text Processing Step
+  - - 1
+- - Data Has Speaker Step
+  - 'no'
+- - Know Speaker Step
+  - 'yes'
+- - Add Speaker Step
+  - sinhalese_speaker
+- - Data Has Language Step
+  - 'no'
+- - Select Language Step
+  - '[und]: my language isn''t here'
+- - Wavs Dir Step
+  - ../regress-si/wavs
+- - Validate Wavs Step
+  - OK
+- - Symbol-Set Step
+  - true
+- - SoX Effects Step
+  - - 0
+    - 1
+    - 2
+- - Dataset Name Step
+  - si
+- - More Datasets Step
+  - 'yes'
+- - Filelist Step
+  - ../regress-xh/line_index.tsv
+- - Dataset Permission Step
+  - Yes, I do have permission to use this data.
+- - Filelist Format Step
+  - tsv
+- - Filelist Has Header Line Step
+  - 'no'
+- - Basename Header Step
+  - 0
+- - Text Header Step
+  - 1
+- - Filelist Text Representation Step
+  - characters
+- - Text Processing Step
+  - - 0
+    - 1
+- - Data Has Speaker Step
+  - 'no'
+- - Know Speaker Step
+  - 'yes'
+- - Add Speaker Step
+  - isixhosa_speaker
+- - Data Has Language Step
+  - 'no'
+- - Select Language Step
+  - '[und]: my language isn''t here'
+- - Wavs Dir Step
+  - ../regress-xh/wavs
+- - Validate Wavs Step
+  - OK
+- - Symbol-Set Step
+  - true
+- - SoX Effects Step
+  - - 1
+- - Dataset Name Step
+  - xh
+- - More Datasets Step
+  - 'no'
+- - Config Format Step
+  - 'yaml'
diff --git a/everyvoice/tests/test_cli.py b/everyvoice/tests/test_cli.py
index 0e1f547a..6577192f 100644
--- a/everyvoice/tests/test_cli.py
+++ b/everyvoice/tests/test_cli.py
@@ -377,7 +377,7 @@ def test_demo_with_bad_args(self):
         self.assertIn("Invalid value", result.output)
 
     def test_create_demo_app_with_errors(self):
-        # outputs is the first thing to get checked, because it's can be done as
+        # outputs is the first thing to get checked, because it can be done as
         # a quick check before loading any models.
         with self.assertRaises(ValueError) as cm:
             create_demo_app(
diff --git a/pyproject.toml b/pyproject.toml
index e2322d4a..0f6d8636 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -94,6 +94,7 @@ torch = [
 dev = [
   "black~=24.3",
   "coverage",
+  "diff-cover",
   "flake8>=4.0.1",
   "gitlint-core>=0.19.0",
   "isort>=5.10.1",
@@ -106,7 +107,7 @@ dev = [
   "types-tqdm>=4.64,<5.0",
   "everyvoice[test]",
 ]
-test = ["jsonschema>=4.17.3", "pep440>=0.1.2"]
+test = ["jsonschema>=4.17.3", "pep440>=0.1.2", "playwright>=1.52.0"]
 docs = [
   "mkdocs>=1.5.2",
   "mkdocs-click>=0.8.0",


Reviews:
user: login: roedoejet
body: Code looks great. Just a couple of minor comments. I haven't tested this as it sounds like @marctessier is. Thanks Eric - really nice to see playwright tests up for this!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-26T20:38:27Z
user: login: joanise
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-26T21:56:47Z
user: login: joanise
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-26T21:57:25Z
user: login: joanise
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-29T20:49:18Z
user: login: joanise
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-30T17:40:38Z

Review comments:
diff_hunk: @@ -0,0 +1,21 @@
+#!/usr/bin/env python
+
+"""
+For some reason, the coverage reports in regression tend to include the full paths.
+This script filters them to just start with everyvoice/... and realigns the header
+and footer accordingly.
+
+Usage: coverage report --include='*/everyvoice/*' | python format-coverage.py
path: everyvoice/tests/regression/format-coverage.py
user: login: roedoejet
body: I think we could do this natively within coverage by putting:

```
[run]
relative_files = true
```

in our .coveragerc config. See https://github.com/nedbat/coveragepy/issues/948
created_at: 2025-05-26T20:32:47Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,53 @@
+"""
+Test the Demo App with the regress-lj-full demo model as some exclusion options.
path: everyvoice/tests/regression/test-demo-app-lj-full.py
user: login: roedoejet
body: I'm not sure I understand this sentence "as some exclusion options" ?
created_at: 2025-05-26T20:35:51Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,21 @@
+#!/usr/bin/env python
+
+"""
+For some reason, the coverage reports in regression tend to include the full paths.
+This script filters them to just start with everyvoice/... and realigns the header
+and footer accordingly.
+
+Usage: coverage report --include='*/everyvoice/*' | python format-coverage.py
path: everyvoice/tests/regression/format-coverage.py
user: login: joanise
body: This was a while back, but I'm pretty sure I tried this and it failed for me. But I should test this again before merging.
created_at: 2025-05-26T21:56:47Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,53 @@
+"""
+Test the Demo App with the regress-lj-full demo model as some exclusion options.
path: everyvoice/tests/regression/test-demo-app-lj-full.py
user: login: joanise
body: Neither do I! I'll have to try to remember what I meant here... Must be a typo.
created_at: 2025-05-26T21:57:25Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,21 @@
+#!/usr/bin/env python
+
+"""
+For some reason, the coverage reports in regression tend to include the full paths.
+This script filters them to just start with everyvoice/... and realigns the header
+and footer accordingly.
+
+Usage: coverage report --include='*/everyvoice/*' | python format-coverage.py
path: everyvoice/tests/regression/format-coverage.py
user: login: joanise
body: OK, figured it out: it works if I run `coverage xml` from the `EveryVoice/everyvoice` directory, but not when I run it from the regression folder. I'll have to do some configuring to use the option without changing where the report ends up.
created_at: 2025-05-29T20:49:18Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,53 @@
+"""
+Test the Demo App with the regress-lj-full demo model as some exclusion options.
path: everyvoice/tests/regression/test-demo-app-lj-full.py
user: login: joanise
body: fixed
created_at: 2025-05-30T17:40:38Z
author_association: MEMBER

PR comments:
user: login: semanticdiff-com[bot]
created_at: 2025-05-02T18:51:21Z
body: Review changes with &nbsp;<a href="https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes"><picture><source media="(prefers-color-scheme: dark)" srcset="https://app.semanticdiff.com/images/github_button.svg?theme=dark&smaller=0"><source media="(prefers-color-scheme: light)" srcset="https://app.semanticdiff.com/images/github_button.svg?theme=light&smaller=0"><img src="https://app.semanticdiff.com/images/github_button.svg?theme=light&smaller=0" align="center" alt="SemanticDiff"></picture></a>
<details>
<summary>Changed Files</summary>
<br/>

| File | Status |
| :--- | :---   |
| [<img src="https://app.semanticdiff.com/static/file-icons/python.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/demo/app.py)&nbsp;&nbsp;[everyvoice/demo/app\.py](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/demo/app.py) | [<img src="https://app.semanticdiff.com/images/progressbar.svg?progress=0">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/demo/app.py) &nbsp;0% smaller |
| [<img src="https://app.semanticdiff.com/static/file-icons/git.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/.gitignore)&nbsp;&nbsp;[everyvoice/tests/regression/\.gitignore](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/.gitignore) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/markdown.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/README.md)&nbsp;&nbsp;[everyvoice/tests/regression/README\.md](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/README.md) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/combine-coverage.sh)&nbsp;&nbsp;[everyvoice/tests/regression/combine\-coverage\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/combine-coverage.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go-gpsc5.sh)&nbsp;&nbsp;[everyvoice/tests/regression/go\-gpsc5\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go-gpsc5.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go-gpscc.sh)&nbsp;&nbsp;[everyvoice/tests/regression/go\-gpscc\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go-gpscc.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go-trixie.sh)&nbsp;&nbsp;[everyvoice/tests/regression/go\-trixie\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go-trixie.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go.sh)&nbsp;&nbsp;[everyvoice/tests/regression/go\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/go.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/prep-datasets.sh)&nbsp;&nbsp;[everyvoice/tests/regression/prep\-datasets\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/prep-datasets.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/regression-test.sh)&nbsp;&nbsp;[everyvoice/tests/regression/regression\-test\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/regression-test.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/run-demo-app-lj-full.sh)&nbsp;&nbsp;[everyvoice/tests/regression/run\-demo\-app\-lj\-full\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/run-demo-app-lj-full.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/console.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/run-demo-app-mix.sh)&nbsp;&nbsp;[everyvoice/tests/regression/run\-demo\-app\-mix\.sh](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/run-demo-app-mix.sh) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/python.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/test-demo-app-lj-full.py)&nbsp;&nbsp;[everyvoice/tests/regression/test\-demo\-app\-lj\-full\.py](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/test-demo-app-lj-full.py) | [<img src="https://app.semanticdiff.com/images/progressbar.svg?progress=0">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/test-demo-app-lj-full.py) &nbsp;0% smaller |
| [<img src="https://app.semanticdiff.com/static/file-icons/python.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/test-demo-app-mix.py)&nbsp;&nbsp;[everyvoice/tests/regression/test\-demo\-app\-mix\.py](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/test-demo-app-mix.py) | [<img src="https://app.semanticdiff.com/images/progressbar.svg?progress=0">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/test-demo-app-mix.py) &nbsp;0% smaller |
| [<img src="https://app.semanticdiff.com/static/file-icons/python.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/wait-for-demo-app.py)&nbsp;&nbsp;[everyvoice/tests/regression/wait\-for\-demo\-app\.py](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/wait-for-demo-app.py) | [<img src="https://app.semanticdiff.com/images/progressbar.svg?progress=0">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/wait-for-demo-app.py) &nbsp;0% smaller |
| [<img src="https://app.semanticdiff.com/static/file-icons/file.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/wizard-resume-mix)&nbsp;&nbsp;[everyvoice/tests/regression/wizard\-resume\-mix](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/regression/wizard-resume-mix) | Unsupported file format |
| [<img src="https://app.semanticdiff.com/static/file-icons/python.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/test_cli.py)&nbsp;&nbsp;[everyvoice/tests/test\_cli\.py](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/test_cli.py) | [<img src="https://app.semanticdiff.com/images/progressbar.svg?progress=0">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#everyvoice/tests/test_cli.py) &nbsp;0% smaller |
| [<img src="https://app.semanticdiff.com/static/file-icons/python-misc.svg" width="20" height="20" align="center">](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#pyproject.toml)&nbsp;&nbsp;[pyproject\.toml](https://app.semanticdiff.com/gh/EveryVoiceTTS/EveryVoice/pull/648/changes#pyproject.toml) | Unsupported file format |

</details>
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-05-02T18:57:47Z
body: <!-- add-pr-comment:add-pr-comment -->

```
CLI load time: 0:00.26
Pull Request HEAD: cb584add3d49330e9a771265e11d9bf4a7664216
Imports that take more than 0.1 s:
import time: self [us] | cumulative | imported package
import time:       276 |     117729 |   typer
import time:      8100 |     197369 | everyvoice.cli

```
author_association: CONTRIBUTOR
user: login: codecov[bot]
created_at: 2025-05-02T18:58:35Z
body: ## [Codecov](https://app.codecov.io/gh/EveryVoiceTTS/EveryVoice/pull/648?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS) Report
Attention: Patch coverage is `0%` with `1 line` in your changes missing coverage. Please review.
> Project coverage is 77.20%. Comparing base [(`49d9c8b`)](https://app.codecov.io/gh/EveryVoiceTTS/EveryVoice/commit/49d9c8bc6200f4b22b81d5d6c54297b22bf4580c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS) to head [(`cb584ad`)](https://app.codecov.io/gh/EveryVoiceTTS/EveryVoice/commit/cb584add3d49330e9a771265e11d9bf4a7664216?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS).
> Report is 10 commits behind head on main.

| [Files with missing lines](https://app.codecov.io/gh/EveryVoiceTTS/EveryVoice/pull/648?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS) | Patch % | Lines |
|---|---|---|
| [everyvoice/demo/app.py](https://app.codecov.io/gh/EveryVoiceTTS/EveryVoice/pull/648?src=pr&el=tree&filepath=everyvoice%2Fdemo%2Fapp.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS#diff-ZXZlcnl2b2ljZS9kZW1vL2FwcC5weQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/EveryVoiceTTS/EveryVoice/pull/648?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS) |

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##             main     #648   +/-   ##
=======================================
  Coverage   77.20%   77.20%           
=======================================
  Files          47       47           
  Lines        3751     3751           
  Branches      528      528           
=======================================
  Hits         2896     2896           
  Misses        745      745           
  Partials      110      110           
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/EveryVoiceTTS/EveryVoice/pull/648?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=EveryVoiceTTS).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE
user: login: marctessier
created_at: 2025-05-26T19:49:44Z
body: currently getting 5 error when we run all tests:

```
======================================================================
FAIL: test_filelist_and_text (model.feature_prediction.FastSpeech2_lightning.fs2.tests.test_cli.SynthesizeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/tessierm/sandbox/TxT2Speech/EveryVoice_pr648/everyvoice/model/feature_prediction/FastSpeech2_lightning/fs2/tests/test_cli.py", line 63, in test_filelist_and_text
    self.assertIn(
AssertionError: 'Got arguments for both text and a filelist - this will only process the text. Please re-run without providing text if you want to run batch synthesis' not found in ''

======================================================================
FAIL: test_no_filelist_nor_text (model.feature_prediction.FastSpeech2_lightning.fs2.tests.test_cli.SynthesizeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/tessierm/sandbox/TxT2Speech/EveryVoice_pr648/everyvoice/model/feature_prediction/FastSpeech2_lightning/fs2/tests/test_cli.py", line 81, in test_no_filelist_nor_text
    self.assertIn("You must define either --text or --filelist", result.stdout)
AssertionError: 'You must define either --text or --filelist' not found in ''

======================================================================
FAIL: test_no_model (model.feature_prediction.FastSpeech2_lightning.fs2.tests.test_cli.SynthesizeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/tessierm/sandbox/TxT2Speech/EveryVoice_pr648/everyvoice/model/feature_prediction/FastSpeech2_lightning/fs2/tests/test_cli.py", line 43, in test_no_model
    self.assertIn("Missing argument 'MODEL_PATH'.", result.stdout)
AssertionError: "Missing argument 'MODEL_PATH'." not found in ''

======================================================================
FAIL: test_align_empty_file (model.aligner.wav2vec2aligner.aligner.tests.test_cli.CLITest) [empty file]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/tessierm/sandbox/TxT2Speech/EveryVoice_pr648/everyvoice/model/aligner/wav2vec2aligner/aligner/tests/test_cli.py", line 51, in test_align_empty_file
    self.assertIn("is empty", result.stdout)
AssertionError: 'is empty' not found in ''

======================================================================
FAIL: test_align_empty_file (model.aligner.wav2vec2aligner.aligner.tests.test_cli.CLITest) [file with only empty lines]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/tessierm/sandbox/TxT2Speech/EveryVoice_pr648/everyvoice/model/aligner/wav2vec2aligner/aligner/tests/test_cli.py", line 60, in test_align_empty_file
    self.assertIn("is empty", result.stdout)
AssertionError: 'is empty' not found in ''

----------------------------------------------------------------------
Ran 220 tests in 69.894s

FAILED (failures=5)

```
Tested on GPSC and also locally with the same results.
author_association: COLLABORATOR
user: login: marctessier
created_at: 2025-05-29T14:24:41Z
body: This looks good now,  No more errors!  Nice work!

```
----------------------------------------------------------------------
Ran 231 tests in 95.540s

OK
============ Finished job 5234311 on Thu May 29 10:21:11 AM EDT 2025 with rc=0
```
author_association: COLLABORATOR