Repository: stac-utils/stac-fastapi
Pull request #655 (2024-04-10 12:28:05)

Adding free-text extension.
**Related Issue(s):**

- #

**Description:**

Adding the [free-text extension](https://github.com/stac-api-extensions/freetext-search)

**PR Checklist:**

- [x] `pre-commit` hooks pass locally
- [x] Tests pass (run `make test`)
- [ ] Documentation has been updated to reflect changes, if applicable, and docs build successfully (run `make docs`)
- [ ] Changes are added to the [CHANGELOG](https://github.com/stac-utils/stac-fastapi/blob/main/CHANGES.md).


rhysrevans3: @m-mohr Could I get your opinion on this I know you've helped with the extension definition.
vincentsarago: where are we with this PR? @rhysrevans3 
rhysrevans3: > where are we with this PR? @rhysrevans3

We're unsure on if basic and advanced search are compatible https://github.com/stac-api-extensions/freetext-search/issues/10 

I think the current options are, update the advanced search to better match with the basic search or separate them into two extensions.
vincentsarago: Dear @rhysrevans3 and @m-mohr I've refactored a bit this extension to:
- make the `conformance_classes` a required argument, because some implementation could only support the `free-text` spec in specific endpoints 
- move the `stac_fastapi.extensions.core` 
- added `tests` 
- assume that the param for both GET/POST are of **string** form (ref: https://github.com/stac-api-extensions/freetext-search/issues/11), I think it's then up to the implementation to parse the parameter

m-mohr: > * assume that the param for both GET/POST are of **string** form

Correct for Advanced, not for Basic. See also https://github.com/stac-api-extensions/freetext-search/issues/11
vincentsarago: I guess I'm going to create 2 Extensions to make this a bit more clear
m-mohr: That's a very good idea!
jonhealy1: There is a draft pr in sfeos related to this but I don't think it gives any new information - https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/227/files
vincentsarago: This is ready for review 

It would be nice to add documentation but this can be done in another PR ;-) 
m-mohr: Was the implementation tested against STAC Browser by any chance?
vincentsarago: > Was the implementation tested against STAC Browser by any chance?

no, but I don't know any backend supporting the free-text spec 😓 
vincentsarago: > Was the implementation tested against STAC Browser by any chance?

@m-mohr I don't really see how the free-text extension is used by STAC-Browser

maybe https://github.com/radiantearth/stac-browser/blob/108947ebe8cbd64c7e288bbaa3c5f81293706d3e/src/components/SearchFilter.vue#L141 seems to point that `q` should be an array 🤷 but then I don't know if it's used in a GET or POST request 


vincentsarago: I'm going to merge this PR (so I can continue on the collection-search which depends on the free-text ext) but feel free to comment and we could still make changes before we do a next release

Patch:
diff --git a/CHANGES.md b/CHANGES.md
index 1cc1773..939471b 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -2,8 +2,14 @@
 
 ## [Unreleased] - TBD
 
+### Changed
+
 * add more openapi metadata in input models [#734](https://github.com/stac-utils/stac-fastapi/pull/734)
 
+### Added
+
+* Add Free-text Extension to third party extensions ([#655](https://github.com/stac-utils/stac-fastapi/pull/655))
+
 ## [3.0.0b2] - 2024-07-09
 
 ### Changed
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index e5326e2..79af024 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -36,6 +36,10 @@ nav:
           - core:
               - module: api/stac_fastapi/extensions/core/index.md
               - context: api/stac_fastapi/extensions/core/context.md
+              - free_text:
+                - module: api/stac_fastapi/extensions/core/free_text/index.md
+                - free_text: api/stac_fastapi/extensions/core/free_text/free_text.md
+                - request: api/stac_fastapi/extensions/core/free_text/request.md
               - filter:
                   - module: api/stac_fastapi/extensions/core/filter/index.md
                   - filter: api/stac_fastapi/extensions/core/filter/filter.md
diff --git a/stac_fastapi/api/stac_fastapi/api/config.py b/stac_fastapi/api/stac_fastapi/api/config.py
index 20a7b4a..74a1c73 100644
--- a/stac_fastapi/api/stac_fastapi/api/config.py
+++ b/stac_fastapi/api/stac_fastapi/api/config.py
@@ -19,6 +19,7 @@ class ApiExtensions(enum.Enum):
     sort = "sort"
     transaction = "transaction"
     aggregation = "aggregation"
+    free_text = "free-text"
 
 
 class AddOns(enum.Enum):
diff --git a/stac_fastapi/extensions/stac_fastapi/extensions/core/__init__.py b/stac_fastapi/extensions/stac_fastapi/extensions/core/__init__.py
index 7e29e1f..fe8b664 100644
--- a/stac_fastapi/extensions/stac_fastapi/extensions/core/__init__.py
+++ b/stac_fastapi/extensions/stac_fastapi/extensions/core/__init__.py
@@ -4,6 +4,7 @@ from .aggregation import AggregationExtension
 from .context import ContextExtension
 from .fields import FieldsExtension
 from .filter import FilterExtension
+from .free_text import FreeTextAdvancedExtension, FreeTextExtension
 from .pagination import PaginationExtension, TokenPaginationExtension
 from .query import QueryExtension
 from .sort import SortExtension
@@ -14,6 +15,8 @@ __all__ = (
     "ContextExtension",
     "FieldsExtension",
     "FilterExtension",
+    "FreeTextExtension",
+    "FreeTextAdvancedExtension",
     "PaginationExtension",
     "QueryExtension",
     "SortExtension",
diff --git a/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/__init__.py b/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/__init__.py
new file mode 100644
index 0000000..53906bc
--- /dev/null
+++ b/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/__init__.py
@@ -0,0 +1,13 @@
+"""Query extension module."""
+
+from .free_text import (
+    FreeTextAdvancedExtension,
+    FreeTextConformanceClasses,
+    FreeTextExtension,
+)
+
+__all__ = [
+    "FreeTextExtension",
+    "FreeTextAdvancedExtension",
+    "FreeTextConformanceClasses",
+]
diff --git a/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/free_text.py b/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/free_text.py
new file mode 100644
index 0000000..8b61b32
--- /dev/null
+++ b/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/free_text.py
@@ -0,0 +1,110 @@
+"""Free-text extension."""
+
+from enum import Enum
+from typing import List, Optional
+
+import attr
+from fastapi import FastAPI
+
+from stac_fastapi.types.extension import ApiExtension
+
+from .request import (
+    FreeTextAdvancedExtensionGetRequest,
+    FreeTextAdvancedExtensionPostRequest,
+    FreeTextExtensionGetRequest,
+    FreeTextExtensionPostRequest,
+)
+
+
+class FreeTextConformanceClasses(str, Enum):
+    """Conformance classes for the Free-Text extension.
+
+    See https://github.com/stac-api-extensions/freetext-search
+
+    """
+
+    # https://github.com/stac-api-extensions/freetext-search?tab=readme-ov-file#basic
+    SEARCH = "https://api.stacspec.org/v1.0.0-rc.1/item-search#free-text"
+    COLLECTIONS = "https://api.stacspec.org/v1.0.0-rc.1/collection-search#free-text"
+    ITEMS = "https://api.stacspec.org/v1.0.0-rc.1/ogcapi-features#free-text"
+
+    # https://github.com/stac-api-extensions/freetext-search?tab=readme-ov-file#advanced
+    SEARCH_ADVANCED = (
+        "https://api.stacspec.org/v1.0.0-rc.1/item-search#advanced-free-text"
+    )
+    COLLECTIONS_ADVANCED = (
+        "https://api.stacspec.org/v1.0.0-rc.1/collection-search#advanced-free-text"
+    )
+    ITEMS_ADVANCED = (
+        "https://api.stacspec.org/v1.0.0-rc.1/ogcapi-features#advanced-free-text"
+    )
+
+
+@attr.s
+class FreeTextExtension(ApiExtension):
+    """Free-text Extension.
+
+    The Free-text extension adds an additional `q` parameter to `/search` requests which
+    allows the caller to perform free-text queries against STAC metadata.
+
+    https://github.com/stac-api-extensions/freetext-search?tab=readme-ov-file#basic
+
+    """
+
+    GET = FreeTextExtensionGetRequest
+    POST = FreeTextExtensionPostRequest
+
+    conformance_classes: List[str] = attr.ib(
+        default=[
+            FreeTextConformanceClasses.SEARCH,
+            FreeTextConformanceClasses.COLLECTIONS,
+            FreeTextConformanceClasses.ITEMS,
+        ]
+    )
+    schema_href: Optional[str] = attr.ib(default=None)
+
+    def register(self, app: FastAPI) -> None:
+        """Register the extension with a FastAPI application.
+
+        Args:
+            app: target FastAPI application.
+
+        Returns:
+            None
+        """
+        pass
+
+
+@attr.s
+class FreeTextAdvancedExtension(ApiExtension):
+    """Free-text Extension.
+
+    The Free-text extension adds an additional `q` parameter to `/search` requests which
+    allows the caller to perform free-text queries against STAC metadata.
+
+    https://github.com/stac-api-extensions/freetext-search?tab=readme-ov-file#advanced
+
+    """
+
+    GET = FreeTextAdvancedExtensionGetRequest
+    POST = FreeTextAdvancedExtensionPostRequest
+
+    conformance_classes: List[str] = attr.ib(
+        default=[
+            FreeTextConformanceClasses.SEARCH_ADVANCED,
+            FreeTextConformanceClasses.COLLECTIONS_ADVANCED,
+            FreeTextConformanceClasses.ITEMS_ADVANCED,
+        ]
+    )
+    schema_href: Optional[str] = attr.ib(default=None)
+
+    def register(self, app: FastAPI) -> None:
+        """Register the extension with a FastAPI application.
+
+        Args:
+            app: target FastAPI application.
+
+        Returns:
+            None
+        """
+        pass
diff --git a/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/request.py b/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/request.py
new file mode 100644
index 0000000..07aa7be
--- /dev/null
+++ b/stac_fastapi/extensions/stac_fastapi/extensions/core/free_text/request.py
@@ -0,0 +1,64 @@
+"""Request model for the Free-text extension."""
+
+from typing import List, Optional
+
+import attr
+from fastapi import Query
+from pydantic import BaseModel, Field
+from typing_extensions import Annotated
+
+from stac_fastapi.types.search import APIRequest, str2list
+
+
+def _ft_converter(
+    val: Annotated[
+        Optional[str],
+        Query(
+            description="Parameter to perform free-text queries against STAC metadata",
+            json_schema_extra={
+                "example": "ocean,coast",
+            },
+        ),
+    ] = None,
+) -> Optional[List[str]]:
+    return str2list(val)
+
+
+@attr.s
+class FreeTextExtensionGetRequest(APIRequest):
+    """Free-text Extension GET request model."""
+
+    q: Optional[List[str]] = attr.ib(default=None, converter=_ft_converter)
+
+
+class FreeTextExtensionPostRequest(BaseModel):
+    """Free-text Extension POST request model."""
+
+    q: Optional[List[str]] = Field(
+        None,
+        description="Parameter to perform free-text queries against STAC metadata",
+    )
+
+
+@attr.s
+class FreeTextAdvancedExtensionGetRequest(APIRequest):
+    """Free-text Extension GET request model."""
+
+    q: Annotated[
+        Optional[str],
+        Query(
+            description="Parameter to perform free-text queries against STAC metadata",
+            json_schema_extra={
+                "example": "ocean,coast",
+            },
+        ),
+    ] = attr.ib(default=None)
+
+
+class FreeTextAdvancedExtensionPostRequest(BaseModel):
+    """Free-text Extension POST request model."""
+
+    q: Optional[str] = Field(
+        None,
+        description="Parameter to perform free-text queries against STAC metadata",
+    )


Test patch:
diff --git a/stac_fastapi/extensions/tests/test_free_text.py b/stac_fastapi/extensions/tests/test_free_text.py
new file mode 100644
index 0000000..55f253a
--- /dev/null
+++ b/stac_fastapi/extensions/tests/test_free_text.py
@@ -0,0 +1,322 @@
+# noqa: E501
+"""test freetext extension."""
+
+
+from starlette.testclient import TestClient
+
+from stac_fastapi.api.app import StacApi
+from stac_fastapi.api.models import (
+    ItemCollectionUri,
+    create_get_request_model,
+    create_post_request_model,
+    create_request_model,
+)
+from stac_fastapi.extensions.core import FreeTextAdvancedExtension, FreeTextExtension
+from stac_fastapi.extensions.core.free_text import FreeTextConformanceClasses
+from stac_fastapi.types.config import ApiSettings
+from stac_fastapi.types.core import BaseCoreClient
+
+
+class DummyCoreClient(BaseCoreClient):
+    def all_collections(self, *args, **kwargs):
+        return kwargs.pop("q", None)
+
+    def get_collection(self, *args, **kwargs):
+        raise NotImplementedError
+
+    def get_item(self, *args, **kwargs):
+        raise NotImplementedError
+
+    def get_search(self, *args, **kwargs):
+        return kwargs.pop("q", None)
+
+    def post_search(self, *args, **kwargs):
+        return args[0].q
+
+    def item_collection(self, *args, **kwargs):
+        return kwargs.pop("q", None)
+
+
+def test_search_free_text_search():
+    """Test search endpoints with free-text ext."""
+    settings = ApiSettings()
+    extensions = [
+        FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.SEARCH])
+    ]
+
+    api = StacApi(
+        settings=settings,
+        client=DummyCoreClient(),
+        extensions=extensions,
+        search_get_request_model=create_get_request_model(extensions),
+        search_post_request_model=create_post_request_model(extensions),
+    )
+    with TestClient(api.app) as client:
+        response = client.get("/conformance")
+        assert response.is_success, response.json()
+        response_dict = response.json()
+        conforms = response_dict["conformsTo"]
+        assert FreeTextConformanceClasses.SEARCH in conforms
+
+        # /search - GET, no free-text
+        response = client.get(
+            "/search",
+            params={"collections": ["test"]},
+        )
+        assert response.is_success
+        assert not response.text
+
+        # /search - GET, free-text option
+        response = client.get(
+            "/search",
+            params={
+                "collections": ["test"],
+                "q": "ocean,coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == ["ocean", "coast"]
+
+        # /search - POST, no free-text
+        response = client.post(
+            "/search",
+            json={
+                "collections": ["test"],
+            },
+        )
+        assert response.is_success
+        assert not response.text
+
+        # /search - POST, free-text option
+        response = client.post(
+            "/search",
+            json={
+                "collections": ["test"],
+                "q": ["ocean", "coast"],
+            },
+        )
+
+        assert response.is_success, response.text
+        assert response.json() == ["ocean", "coast"]
+
+
+def test_search_free_text_complete():
+    """Test search,collections,items endpoints with free-text ext."""
+    settings = ApiSettings()
+
+    free_text = FreeTextExtension(
+        conformance_classes=[
+            FreeTextConformanceClasses.SEARCH,
+            FreeTextConformanceClasses.ITEMS,
+            FreeTextConformanceClasses.COLLECTIONS,
+        ]
+    )
+
+    search_get_model = create_get_request_model([free_text])
+    search_post_model = create_post_request_model([free_text])
+    items_get_model = create_request_model(
+        "ItemCollectionURI",
+        base_model=ItemCollectionUri,
+        mixins=[free_text.GET],
+    )
+
+    api = StacApi(
+        settings=settings,
+        client=DummyCoreClient(),
+        extensions=[free_text],
+        search_get_request_model=search_get_model,
+        search_post_request_model=search_post_model,
+        collections_get_request_model=free_text.GET,
+        items_get_request_model=items_get_model,
+    )
+    with TestClient(api.app) as client:
+        response = client.get("/conformance")
+        assert response.is_success, response.json()
+        response_dict = response.json()
+        conforms = response_dict["conformsTo"]
+        assert FreeTextConformanceClasses.SEARCH in conforms
+        assert FreeTextConformanceClasses.ITEMS in conforms
+        assert FreeTextConformanceClasses.COLLECTIONS in conforms
+
+        # /search - GET, no free-text
+        response = client.get(
+            "/search",
+            params={"collections": ["test"]},
+        )
+        assert response.is_success
+        assert not response.text
+
+        # /search - GET, free-text option
+        response = client.get(
+            "/search",
+            params={
+                "collections": ["test"],
+                "q": "ocean,coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == ["ocean", "coast"]
+
+        # /collections - GET, free-text option
+        response = client.get(
+            "/collections",
+            params={
+                "q": "ocean,coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == ["ocean", "coast"]
+
+        # /items - GET, free-text option
+        response = client.get(
+            "/collections/test/items",
+            params={
+                "q": "ocean,coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == ["ocean", "coast"]
+
+
+def test_search_free_text_search_advanced():
+    """Test search endpoints with free-text ext."""
+    settings = ApiSettings()
+    extensions = [
+        FreeTextAdvancedExtension(
+            conformance_classes=[FreeTextConformanceClasses.SEARCH_ADVANCED]
+        )
+    ]
+
+    api = StacApi(
+        settings=settings,
+        client=DummyCoreClient(),
+        extensions=extensions,
+        search_get_request_model=create_get_request_model(extensions),
+        search_post_request_model=create_post_request_model(extensions),
+    )
+    with TestClient(api.app) as client:
+        response = client.get("/conformance")
+        assert response.is_success, response.json()
+        response_dict = response.json()
+
+        conforms = response_dict["conformsTo"]
+        assert FreeTextConformanceClasses.SEARCH_ADVANCED in conforms
+
+        # /search - GET, no free-text
+        response = client.get(
+            "/search",
+            params={"collections": ["test"]},
+        )
+        assert response.is_success
+        assert not response.text
+
+        # /search - GET, free-text option
+        response = client.get(
+            "/search",
+            params={
+                "collections": ["test"],
+                "q": "+ocean,-coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == "+ocean,-coast"
+
+        # /search - POST, no free-text
+        response = client.post(
+            "/search",
+            json={
+                "collections": ["test"],
+            },
+        )
+        assert response.is_success
+        assert not response.text
+
+        # /search - POST, free-text option
+        response = client.post(
+            "/search",
+            json={
+                "collections": ["test"],
+                "q": "+ocean,-coast",
+            },
+        )
+
+        assert response.is_success, response.text
+        assert response.json() == "+ocean,-coast"
+
+
+def test_search_free_text_advanced_complete():
+    """Test search,collections,items endpoints with free-text ext."""
+    settings = ApiSettings()
+
+    free_text = FreeTextAdvancedExtension(
+        conformance_classes=[
+            FreeTextConformanceClasses.SEARCH_ADVANCED,
+            FreeTextConformanceClasses.ITEMS_ADVANCED,
+            FreeTextConformanceClasses.COLLECTIONS_ADVANCED,
+        ]
+    )
+
+    search_get_model = create_get_request_model([free_text])
+    search_post_model = create_post_request_model([free_text])
+    items_get_model = create_request_model(
+        "ItemCollectionURI",
+        base_model=ItemCollectionUri,
+        mixins=[free_text.GET],
+    )
+
+    api = StacApi(
+        settings=settings,
+        client=DummyCoreClient(),
+        extensions=[free_text],
+        search_get_request_model=search_get_model,
+        search_post_request_model=search_post_model,
+        collections_get_request_model=free_text.GET,
+        items_get_request_model=items_get_model,
+    )
+    with TestClient(api.app) as client:
+        response = client.get("/conformance")
+        assert response.is_success, response.json()
+        response_dict = response.json()
+        conforms = response_dict["conformsTo"]
+        assert FreeTextConformanceClasses.SEARCH_ADVANCED in conforms
+        assert FreeTextConformanceClasses.ITEMS_ADVANCED in conforms
+        assert FreeTextConformanceClasses.COLLECTIONS_ADVANCED in conforms
+
+        # /search - GET, no free-text
+        response = client.get(
+            "/search",
+            params={"collections": ["test"]},
+        )
+        assert response.is_success
+        assert not response.text
+
+        # /search - GET, free-text option
+        response = client.get(
+            "/search",
+            params={
+                "collections": ["test"],
+                "q": "ocean,coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == "ocean,coast"
+
+        # /collections - GET, free-text option
+        response = client.get(
+            "/collections",
+            params={
+                "q": "ocean,coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == "ocean,coast"
+
+        # /items - GET, free-text option
+        response = client.get(
+            "/collections/test/items",
+            params={
+                "q": "ocean,coast",
+            },
+        )
+        assert response.is_success, response.text
+        assert response.json() == "ocean,coast"