Repository: tlambert03/fpbasepy

Issue:
Various strings from fpbase.list_fluorophores() throw Validation Errors when attempting to fetch default state
* fpbasepy version: 0.1.0
* Python version: 3.10.12
* Operating System: Ubuntu 22.04

### Description

Was attempting to programatically fetch spectra from fpbase; as part of testing that any spectra could be arbitrarily fetched ran into a few problems. Some strings from the list of fluorophores throw errors. I attach the list of fluorophores that led to problematic outcomes.

### What I Did

```
Paste the command(s) you ran and the output.

spectra = np.zeros([len(dye_names), len(wavelength)])
for i, dye_name in enumerate(dye_names):
    try:
        dye_object = fpbase.get_fluorophore(dye_name).default_state
        if dye_object is not None:
            list_sorted = [x for x in dye_object.spectra if "EM" in x.subtype]
            if len(list_sorted) > 0:
                dye_rawdata = np.array(list_sorted[0].data)
                dye_rawwavelength = dye_rawdata[:, 0]
                dye_rawspectrum = dye_rawdata[:, 1]
                dye_rescaled = np.interp(
                        x=wavelength,
                        xp=dye_rawwavelength,
                        fp=dye_rawspectrum,
                        left=0,
                        right=0,
                    )
                spectra[i, :] = dye_rescaled / np.sum(dye_rescaled)
    except:
        print(dye_name)
        continue

dye names that led to problems:
6C
Clover1.5
EYFP-F46L
K-GECO1
MemBrite Fix 488/515
O-GECO1
PROSS-eGFP
Q80R
R-GECO1
R-GECO1.2
REX-GECO1
RSGFP1
RSGFP2
RSGFP3
RSGFP4
RSGFP6
RSGFP7
dClover2 A206K
htFuncLib_acid.1
htFuncLib_acid.2
htFuncLib_acid.3
htFuncLib_acid.4
htFuncLib_acid.5
htFuncLib_acid.6
htFuncLib_basic.1
htFuncLib_basic.2
htFuncLib_basic.3
htFuncLib_basic.4
htFuncLib_basic.6
htFuncLib_bleach.1
htFuncLib_bleach.2
htFuncLib_bleach.3
htFuncLib_bleach.4
htFuncLib_bleach.5
htFuncLib_bleach.6
htFuncLib_em-blue-shift.1
htFuncLib_em-blue-shift.2
htFuncLib_fast.1
htFuncLib_fast.2
htFuncLib_fast.3
htFuncLib_fast.4
htFuncLib_mid.10
htFuncLib_mid.11
htFuncLib_mid.12
htFuncLib_mid.13
htFuncLib_mid.2
htFuncLib_mid.3
htFuncLib_mid.4
htFuncLib_mid.5
htFuncLib_mid.6
htFuncLib_mid.7
htFuncLib_mid.8
htFuncLib_mid.9
htFuncLib_photostable.1
htFuncLib_photostable.2
htFuncLib_sf:acid.3
htFuncLib_sf:acid.4
htFuncLib_sf:acid.5
htFuncLib_sf:bleach.1
htFuncLib_sf:bleach.2
htFuncLib_sf:bleach.3
htFuncLib_sf:em-blue-shift.1
htFuncLib_sf:em-blue-shift.2
htFuncLib_sf:fast.3
htFuncLib_sf:mid.10
htFuncLib_sf:mid.11
htFuncLib_sf:mid.12
htFuncLib_sf:mid.5
htFuncLib_sf:mid.7
htFuncLib_sf:mid.9
htFuncLib_sf:photostable.1
htFuncLib_sf:photostable.2
htFuncLib_sf:thermo.2
htFuncLib_sf:thermo.9
htFuncLib_slow.1
htFuncLib_slow.2
htFuncLib_slow.3
htFuncLib_slow.4
htFuncLib_slow.5
htFuncLib_thermo.1
htFuncLib_thermo.2
htFuncLib_thermo.3
htFuncLib_thermo.4
htFuncLib_thermo.5
htFuncLib_thermo.6
htFuncLib_thermo.8
htFuncLib_thermo.9
jREX-GECO1
mClover1.5
mEos2-NA
mOFP.T.12
mOFP.T.8
mRFP1.5
mTurquoise-146G
mTurquoise-146S
mTurquoise-DR
mTurquoise-GL
mTurquoise-GV
mTurquoise-RA

Traceback:
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[81], line 1
----> 1 fpbase.get_fluorophore('O-GECO1')

File ~/.virtualenvs/pyBayerSMLM/lib/python3.10/site-packages/fpbase/_fetch.py:232, in get_fluorophore(name)
    231 def get_fluorophore(name: str) -> Fluorophore:
--> 232     return FPbaseClient.instance().get_fluorophore(name)

File ~/.virtualenvs/pyBayerSMLM/lib/python3.10/site-packages/fpbase/_fetch.py:79, in FPbaseClient.get_fluorophore(self, name)
     77     return self._get_dye_by_id(fluor_info["id"])
     78 elif fluor_info["type"] == "p":
---> 79     return self._get_protein_by_id(fluor_info["id"])
     80 raise ValueError(  # pragma: no cover
     81     f"Invalid fluorophore type {fluor_info['type']!r}"
     82 )

File ~/.virtualenvs/pyBayerSMLM/lib/python3.10/site-packages/fpbase/_fetch.py:220, in FPbaseClient._get_protein_by_id(self, id)
    218 def _get_protein_by_id(self, id: str) -> Protein:
    219     resp = self._send_query(PROTEIN_QUERY, {"id": id})
--> 220     return ProteinResponse.model_validate_json(resp).data.protein

File ~/.virtualenvs/pyBayerSMLM/lib/python3.10/site-packages/pydantic/main.py:656, in BaseModel.model_validate_json(cls, json_data, strict, context)
    654 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    655 __tracebackhide__ = True
--> 656 return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context)

ValidationError: 1 validation error for ProteinResponse
data.protein.pdb
  Input should be a valid array [type=list_type, input_value=None, input_type=NoneType]
    For further information visit https://errors.pydantic.dev/2.10/v/list_type
```

Hints from issue comments:
@tlambert03: thanks @JBeckwith, fixed in #2, will push v0.1.1 in a moment

Solution patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index fc1fa5f..b0f11e0 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -14,13 +14,13 @@ repos:
       - id: validate-pyproject
 
   - repo: https://github.com/crate-ci/typos
-    rev: typos-dict-v0.11.35
+    rev: typos-dict-v0.11.37
     hooks:
       - id: typos
         args: [--force-exclude]  # omitting --write-changes
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.7.4
+    rev: v0.8.3
     hooks:
       - id: ruff
         args: [--fix]  # may also add '--unsafe-fixes'
diff --git a/src/fpbase/models.py b/src/fpbase/models.py
index 27fbf6c..7967a9b 100644
--- a/src/fpbase/models.py
+++ b/src/fpbase/models.py
@@ -1,13 +1,33 @@
 """Main fetching logic."""
 
+from collections.abc import Sequence
 from enum import Enum
-from typing import TYPE_CHECKING, Any, Optional
+from typing import TYPE_CHECKING, Annotated, Any, Optional, TypeVar
 
-from pydantic import BaseModel, Field, computed_field, model_validator
+from pydantic import (
+    BaseModel,
+    BeforeValidator,
+    Field,
+    computed_field,
+    model_validator,
+)
 
 if TYPE_CHECKING:
     from collections.abc import Iterable
 
+
+def _null_to_list(v: Any) -> list:
+    if v is None:
+        return []
+    elif isinstance(v, Sequence):
+        return list(v)
+    raise ValueError(f"Expected a list or None, got {v!r}")  # pragma: no cover
+
+
+T = TypeVar("T")
+# type that accepts null as json input and returns an empty list
+SafeList = Annotated[list[T], BeforeValidator(_null_to_list)]
+
 __all__ = [
     "Filter",
     "FilterPlacement",
@@ -135,7 +155,7 @@ class State(BaseModel):
     exhex: str = ""
     ext_coeff: Optional[float] = Field(None, alias="extCoeff")  # M^-1 cm^-1
     qy: Optional[float] = None
-    spectra: list[Spectrum] = Field(default_factory=list)
+    spectra: SafeList[Spectrum] = Field(default_factory=list)
     lifetime: Optional[float] = None  # ns
 
     @property
@@ -158,7 +178,7 @@ class Fluorophore(BaseModel):
     name: str
     id: str
     default_state: Optional[State] = Field(None, alias="defaultState")
-    states: list[State] = Field(default_factory=list)
+    states: SafeList[State] = Field(default_factory=list)
 
     @model_validator(mode="before")
     @classmethod
@@ -193,14 +213,14 @@ def url(self) -> str:
 
 class Protein(Fluorophore):
     seq: Optional[str] = None
-    pdb: list[str] = Field(default_factory=list)
+    pdb: SafeList[str] = Field(default_factory=list)
     genbank: Optional[str] = None
     uniprot: Optional[str] = None
     agg: Optional[Olig] = None
     switch_type: Optional[SwitchType] = Field(None, alias="switchType")
     primary_reference: Optional[Reference] = Field(None, alias="primaryReference")
-    references: list[Reference] = Field(default_factory=list)
-    states: list[State] = Field(default_factory=list)
+    references: SafeList[Reference] = Field(default_factory=list)
+    states: SafeList[State] = Field(default_factory=list)
     # default_state: Optional[State] = Field(None, alias="defaultState")
 
 
@@ -216,7 +236,7 @@ class OpticalConfig(BaseModel):
     """A collection of filters and light sources."""
 
     name: str
-    filters: list[FilterPlacement]
+    filters: SafeList[FilterPlacement]
     camera: Optional["Camera"]
     light: Optional["LightSource"]
     laser: Optional[int]
@@ -227,7 +247,7 @@ class Microscope(BaseModel):
 
     id: str
     name: str
-    opticalConfigs: list[OpticalConfig]
+    opticalConfigs: SafeList[OpticalConfig]
 
 
 class _MicroscopePayload(BaseModel):


Test patch:
diff --git a/tests/test_fpbase.py b/tests/test_fpbase.py
index c76e63e..fde2c5f 100644
--- a/tests/test_fpbase.py
+++ b/tests/test_fpbase.py
@@ -90,3 +90,8 @@ def test_generic_gql_query() -> None:
     q = "query getProtein($id: String!){ protein(id: $id){ name } }"
     data = fpbase.graphql_query(q, {"id": "R9NL8"})
     assert data["data"]["protein"]["name"] == "EGFP"
+
+
+@pytest.mark.parametrize("name", ["Clover1.5", "6C", "dClover2 A206K"])
+def test_fluors_with_no_pdb(name: str) -> None:
+    fpbase.get_fluorophore(name)