Repository: fieldsoftheworld/ftw-baselines (language: python)

Issue:
Add basic tests for CLI
We should add some very basic tests that run all commands step by step on a small sample to check we don't break anything.

- [x] Test framework
- [x] Enable CI
- [x] Basic tests for ftw data
- [x] Basic tests for ftw inference
- [x] Basic tests for ftw model
Polygonize potential improvements
These should perhaps break out into their own issues, but a number of these could likely be done in one chunk of work for the 'polygonize' command proposed in #19 (these suggestions are currently based on #25 where it is its own command).

- [x] Don't require output_fn - default to the same name / location as input, plus the right file format suffix.
- [x] Select file format (at least GeoParquet, GeoJSON & GeoPackage)
- [x] Automatically use the right file format based on suffix of input file name.
- [ ] ~Fiboa output - determination_method = auto-imagery, plus ideally pick-up determination_datetime from input files (depends on #26)~ -> #53
- [x] Calculate and include area (and perimeter?) - default to true, but be able to turn off.
- [ ] ~Whether to use fiboa or just write minimal attribute fields (default to using fiboa)~ -> #53 
- [x] Logic for warnings specific to when you want to polygonize
- [ ] ~Control over the window size for the grid?~ -> #54
- [ ] ~Get rid of the "arbitrarily split" polygons from the grid step (ie dissolve and split into single parts as a very final step). Potentially expose as option.~ -> #54 
- [ ] #55
- [ ] Option to reproject? Like to put the output into EPSG:4326
- [x] Option to remove fields below X size, to filter out the super small areas that sometimes show up. Maybe default to ~1 pixel size, and then let people increase it.
- [x] Option to close holes: https://github.com/fieldsoftheworld/ftw-baselines/pull/31/files#r1815997140 -> #39 
Add basic tests for CLI
We should add some very basic tests that run all commands step by step on a small sample to check we don't break anything.

- [x] Test framework
- [x] Enable CI
- [x] Basic tests for ftw data
- [x] Basic tests for ftw inference
- [x] Basic tests for ftw model


Discussion / hints:

@m-mohr - could you take on a chunk of these and then we can make additional issues for the ones you don't do.

Top one to me is fiboa / geoparquet output. And could do json too in fiboa, and then make geopackage 'fiboa-like' by default. And make it so if someone says -o fields.parquet then it just automatically does fiboa and geoparquet.


Fix patch:
diff --git a/.gitignore b/.gitignore
index 9fad49f..546100d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,3 +12,4 @@ src/ftw.egg-info/
 *.tif
 .pytest_cache/
 /results.csv
+/austria_example_output_full.gpkg


Test patch:
diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml
index 50837c7..78d3dcd 100644
--- a/.github/workflows/pytest.yml
+++ b/.github/workflows/pytest.yml
@@ -23,5 +23,7 @@ jobs:
           auto-activate-base: false
       - name: Install Python dependencies
         run: pip install -e .[dev]
+      - name: Download model
+        run: wget https://github.com/fieldsoftheworld/ftw-baselines/releases/download/Pretrained-Models/3_Class_FULL_FTW_Pretrained.ckpt
       - name: Execute test suites
         run: pytest src/tests
diff --git a/src/tests/test_inference.py b/src/tests/test_inference.py
index a1831f7..03ce191 100644
--- a/src/tests/test_inference.py
+++ b/src/tests/test_inference.py
@@ -1,8 +1,12 @@
 
+import os
+
 from click.testing import CliRunner
 
 from ftw_cli.cli import inference_download, inference_polygonize, inference_run
 
+INF_IMG_PATH = "inference_imagery/austria_example.tif"
+INF_OUT_PATH = "austria_example_output_full.tif"
 
 def test_inference_download(): # create_input
     runner = CliRunner()
@@ -12,24 +16,60 @@ def test_inference_download(): # create_input
     assert result.exit_code == 0, result.output
     assert "Usage: download [OPTIONS]" in result.output
 
-    # TODO: Add more tests
+    # Download imagery by ID from MS Planetary Computer
+    result = runner.invoke(inference_download, [
+        "--win_a=S2B_MSIL2A_20210617T100559_R022_T33UUP_20210624T063729",
+        "--win_b=S2B_MSIL2A_20210925T101019_R022_T33UUP_20210926T121923",
+        "-o", INF_IMG_PATH,
+        "-f"
+    ])
+    assert result.exit_code == 0, result.output
+    assert "Finished saving window A to file" in result.output
+    assert "Finished saving window B to file" in result.output
+    assert "Finished merging and writing output" in result.output
+    assert os.path.exists(INF_IMG_PATH)
 
-def test_inference_polygonize():
+def test_inference_run():
     runner = CliRunner()
 
     # Check help
-    result = runner.invoke(inference_polygonize, ["--help"])
+    result = runner.invoke(inference_run, ["--help"])
     assert result.exit_code == 0, result.output
-    assert "Usage: polygonize [OPTIONS] INPUT" in result.output
+    assert "Usage: run [OPTIONS] INPUT" in result.output
 
-    # TODO: Add more tests
+    # Run inference
+    model_path = "3_Class_FULL_FTW_Pretrained.ckpt"
+    assert os.path.exists(model_path)
+    result = runner.invoke(inference_run, [
+        INF_IMG_PATH,
+        "--model", model_path,
+        "--out", INF_OUT_PATH,
+        "--gpu", "0",
+        "--overwrite",
+        "--resize_factor", "2",
+        "--overwrite"
+    ])
+    assert result.exit_code == 0, result.output
+    assert "Using custom trainer" in result.output
+    assert "100%|" in result.output
+    assert "Finished inference and saved output" in result.output
+    assert os.path.exists(INF_IMG_PATH)
 
-def test_inference_run():
+
+def test_inference_polygonize():
     runner = CliRunner()
 
     # Check help
-    result = runner.invoke(inference_run, ["--help"])
+    result = runner.invoke(inference_polygonize, ["--help"])
     assert result.exit_code == 0, result.output
-    assert "Usage: run [OPTIONS] INPUT" in result.output
+    assert "Usage: polygonize [OPTIONS] INPUT" in result.output
 
-    # TODO: Add more tests
+    # TODO: Activate once https://github.com/fieldsoftheworld/ftw-baselines/issues/27#issuecomment-2445119067 has been fixed
+    # Polygonize the file
+    # out_path = "austria_example_output_full.gpkg"
+    # result = runner.invoke(inference_polygonize, [INF_OUT_PATH, "-o", out_path, "-f"])
+    # assert result.exit_code == 0, result.output
+    # assert "Polygonizing input file:" in result.output
+    # assert "100%|" in result.output
+    # assert "Finished polygonizing output" in result.output
+    # assert os.path.exists(out_path)