Repository: cogent3/cogent3
Pull request #2329 (2025-05-23 01:59:56)

Modifications arising from updating docs
## Summary by Sourcery

Improve MolType definition and sequence rendering logic, refine type coercion, add tests for custom MolType and HTML output, and update documentation dependencies.

Enhancements:
- Simplify MolType complements and pairing_rules types and default to identity complement for undefined characters
- Refactor MolType alphabet variant fields to be lazily initialized and optional
- Enhance Sequence.to_html to correctly detect leading gaps and handle hyphens in regex
- Extend internal type coercion for nested tuple and list types

Build:
- Add 'cogent3-h5seqs' and 'piqtree' to documentation dependencies in pyproject.toml

Documentation:
- Update multiple RST files under 'doc' to reflect documentation changes

Tests:
- Add tests for creating custom MolType and verifying HTML output of sequences


Patch:
diff --git a/doc/api/index.rst b/doc/api/index.rst
index 82bd1c63f..e9e0f3c2a 100644
--- a/doc/api/index.rst
+++ b/doc/api/index.rst
@@ -71,7 +71,7 @@ These are all top level imports. For example,
 
 .. code-block:: python
 
-    from cogent3 import get_code
+    from cogent3 import available_codes
 
 .. toctree::
     :maxdepth: 1
diff --git a/doc/app/index.rst b/doc/app/index.rst
index 39f07ac83..fdbb83e96 100644
--- a/doc/app/index.rst
+++ b/doc/app/index.rst
@@ -1,8 +1,10 @@
+.. _app-docs:
+
 ####
 apps
 ####
 
-Pre-defined "apps" simplify complex tasks. They provide capabilities that can be used by themselves, or added together to define a pipeline. Find out more about apps below!
+Pre-defined "apps" simplify complex tasks. They provide capabilities that can be used by themselves or added together to define a pipeline. Find out more about apps below!
 
 .. grid::
 
diff --git a/doc/cookbook/annotation_db.rst b/doc/cookbook/annotation_db.rst
index e9c6356ec..556eb78c8 100644
--- a/doc/cookbook/annotation_db.rst
+++ b/doc/cookbook/annotation_db.rst
@@ -33,7 +33,7 @@ How to load an standalone ``AnnotationDb`` from a data file
 
 Typically, we want to load a collection of features from a genomic annotation file, such as a GFF or Genbank file. For the following examples, we will use data from the bacterium *Mycoplasma genitalium*.
 
-.. note:: See the list of :ref:`data_links` to download the data used in the following examples.
+.. note:: See the list of :ref:`data-links` to download the data used in the following examples.
 
 From a GFF file
 """""""""""""""
diff --git a/doc/cookbook/building_phylogenies.rst b/doc/cookbook/building_phylogenies.rst
index bc0ebbade..86754a45b 100644
--- a/doc/cookbook/building_phylogenies.rst
+++ b/doc/cookbook/building_phylogenies.rst
@@ -31,6 +31,21 @@ Directly via ``alignment.quick_tree()``
     tree = tree.balanced()  # purely for display
     print(tree.ascii_art())
 
+Directly via ``alignment.quick_tree()`` with a third-party hook
+===============================================================
+
+You can use the IQ-TREE rapid-NJ algorithm for ``quick_tree()`` by installing `piqtree <https://piqtree.readthedocs.io/en/latest/>`_ and setting ``use_hook="piqtree"``.
+
+.. jupyter-execute::
+
+    from cogent3 import load_aligned_seqs
+
+    aln = load_aligned_seqs("data/primate_brca1.fasta", moltype="dna")
+    tree = aln.quick_tree(calc="TN93", use_hook="piqtree")
+    # dropping branch lengths to simplify display
+    dnd = tree.get_figure(contemporaneous=True, width=600, height=600)
+    dnd.show()
+
 Using the ``DistanceMatrix`` object
 -----------------------------------
 
diff --git a/doc/cookbook/features.rst b/doc/cookbook/features.rst
index 8b0777cc5..353b4b911 100644
--- a/doc/cookbook/features.rst
+++ b/doc/cookbook/features.rst
@@ -143,7 +143,7 @@ How to load bulk Features from a File
 
 Typically, we want to load bulk features from a genomic annotation file, such as a GFF or Genbank file. For the following examples, we will use *Caenorhabditis elegans* chromosome I.
 
-.. note:: See the list of :ref:`data_links` to download the data used in the following examples.
+.. note:: See the list of :ref:`data-links` to download the data used in the following examples.
 
 To load features from a genomic annotation file along with the corresponding sequence, we can use the ``load_seq`` function. The features are stored in a ``AnnotationDb`` and assigned to the ``annotation_db`` attribute of the sequence.
 
diff --git a/doc/cookbook/loading_sequences.rst b/doc/cookbook/loading_sequences.rst
index bee6411fa..3bd1efc6a 100755
--- a/doc/cookbook/loading_sequences.rst
+++ b/doc/cookbook/loading_sequences.rst
@@ -248,3 +248,80 @@ The FASTA label field is frequently overloaded, with different information field
         print(name)
         print(name.info.gi)
         print(name.info.description)
+
+.. _storage-plugin:
+
+Using a third-party plugin for sequence storage
+-----------------------------------------------
+
+Sequence collections and alignments have a ``.storage`` attribute which holds the underlying sequence data and provides basic functions for obtaining it. Users can install a third-party plugin which is customized for different types of sequence data. The following examples require you install the ``cogent3-h5seqs`` plugin. This project provides alternative storage for both unaligned sequences and for alignments.
+
+.. code-block:: shell
+
+    $ pip install cogent3-h5seqs
+
+Selecting an alternate storage backend
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Specify the storage using the ``storage_backend`` argument.
+
+.. jupyter-execute::
+    :raises:
+
+    from cogent3 import load_aligned_seqs
+
+    aln = load_aligned_seqs(
+        "data/long_testseqs.fasta", moltype="dna", storage_backend="h5seqs_aligned"
+    )
+    aln
+
+That's it!
+
+.. jupyter-execute::
+
+    type(aln.storage)
+
+For the ``cogent3-h5seqs`` package you specify a different storage backend for unaligned sequences.
+
+.. jupyter-execute::
+    :raises:
+
+    from cogent3 import load_unaligned_seqs
+
+    seqs = load_unaligned_seqs(
+        "data/long_testseqs.fasta", moltype="dna", storage_backend="h5seqs_unaligned"
+    )
+    type(seqs.storage)
+
+Set the default storage
+^^^^^^^^^^^^^^^^^^^^^^^
+
+You can set the default storage process-wide, so you don't need to use the ``storage_backend`` argument.
+
+.. jupyter-execute::
+    :raises:
+
+    import cogent3
+
+    cogent3.set_storage_defaults(
+        unaligned_seqs="h5seqs_unaligned", aligned_seqs="h5seqs_aligned"
+    )
+
+    aln = cogent3.get_dataset("brca1")
+    type(aln.storage)
+
+When you apply operations, the new backend storage setting is applied.
+
+.. jupyter-execute::
+    :raises:
+
+    coll = aln.degap()
+    type(coll.storage)
+
+
+.. note:: To revert to the ``cogent3`` defaults, use the ``reset`` argument.
+
+    .. jupyter-execute::
+        :raises:
+
+        cogent3.set_storage_defaults(reset=True)
diff --git a/doc/cookbook/moltypes.rst b/doc/cookbook/moltypes.rst
index 520cd4ed0..f75824582 100644
--- a/doc/cookbook/moltypes.rst
+++ b/doc/cookbook/moltypes.rst
@@ -84,19 +84,22 @@ Verify sequences
 Making a custom ``MolType``
 ===========================
 
-We demonstrate this by customising DNA so it allows ``.`` as gaps
+We demonstrate this by customising DNA so it allows a ``.`` as a gap character.
 
 .. jupyter-execute::
 
-    from cogent3.core import moltype as mt
-
-    DNAgapped = mt.MolType(
-        seq_constructor=mt.DnaSequence,
-        motifset=mt.IUPAC_DNA_chars,
-        ambiguities=mt.IUPAC_DNA_ambiguities,
-        complements=mt.IUPAC_DNA_ambiguities_complements,
-        pairs=mt.DnaStandardPairs,
-        gaps=".",
-    )
-    seq = DNAgapped.make_seq("ACG.")
+    from cogent3.core import new_moltype, new_sequence
+
+    mt = new_moltype.MolType(
+            monomers="".join(new_moltype.IUPAC_DNA_chars),
+            ambiguities=new_moltype.IUPAC_DNA_ambiguities,
+            name="dna.gap",
+            complements=new_moltype.IUPAC_DNA_ambiguities_complements,
+            make_seq=new_sequence.DnaSequence,
+            pairing_rules=new_moltype.DNA_STANDARD_PAIRS,
+            mw_calculator=new_moltype.DnaMW,
+            coerce_to=new_moltype.coerce_to_dna,
+            gap=".",
+        )
+    seq = mt.make_seq(seq="ACG.")
     seq
diff --git a/doc/data_file_links.rst b/doc/data_file_links.rst
index f6f741691..8859faf6c 100644
--- a/doc/data_file_links.rst
+++ b/doc/data_file_links.rst
@@ -1,4 +1,4 @@
-.. _data_links:
+.. _data-links:
 
 Data Files Used in the Documentation
 ====================================
diff --git a/doc/index.rst b/doc/index.rst
index 7aaf53050..9b9a3b220 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -1,8 +1,10 @@
+.. _user-guides:
+
 ###########
 User Guides
 ###########
 
-Our documentation shows code examples that have been computed within Jupyter notebooks during the documentation build. This shows what you would see if you were to do these examples yourself, interactively, in Jupyter_. To do that, you will need to :ref:`install cogent3 <install>` and download the individual data files used in the User Guides from :ref:`here <data_links>`.
+Our documentation shows code examples that have been computed within Jupyter notebooks during the documentation build. This shows what you would see if you were to do these examples yourself, interactively, in Jupyter_. To do that, you will need to :ref:`install cogent3 <install>` and download the individual data files used in the User Guides from :ref:`here <data-links>`.
 
 .. grid::
 
diff --git a/pyproject.toml b/pyproject.toml
index 88e6aa0f9..40ec97f52 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -66,6 +66,7 @@ test = [
 doc  = [
     "click",
     "cogent3[extra]",
+    "cogent3-h5seqs",
     "jupyter-sphinx",
     "jupyter_client",
     "jupytext",
@@ -73,6 +74,7 @@ doc  = [
     "nbformat",
     "nbsphinx",
     "numpydoc",
+    "piqtree",
     "sphinx",
     "sphinx-autobuild",
     "pydata_sphinx_theme",
diff --git a/src/cogent3/core/new_alphabet.py b/src/cogent3/core/new_alphabet.py
index 46cfadc4f..1e4986ed1 100644
--- a/src/cogent3/core/new_alphabet.py
+++ b/src/cogent3/core/new_alphabet.py
@@ -47,12 +47,12 @@ def _(orig: bytearray, text: str) -> StrORBytes:
 
 @_coerce_to_type.register
 def _(orig: tuple, text: str) -> StrORBytes:
-    return (text.encode("utf8"),)
+    return _coerce_to_type(orig[0], text)
 
 
 @_coerce_to_type.register
 def _(orig: list, text: str) -> StrORBytes:
-    return (text.encode("utf8"),)
+    return _coerce_to_type(orig[0], text)
 
 
 class AlphabetError(TypeError): ...
diff --git a/src/cogent3/core/new_moltype.py b/src/cogent3/core/new_moltype.py
index 1dc98888e..ebebff8e7 100644
--- a/src/cogent3/core/new_moltype.py
+++ b/src/cogent3/core/new_moltype.py
@@ -421,19 +421,19 @@ class MolType:
     make_seq: dataclasses.InitVar[type]
     gap: OptStr = IUPAC_gap
     missing: OptStr = IUPAC_missing
-    complements: dataclasses.InitVar[dict[str, frozenset[str]] | None] = None
+    complements: dataclasses.InitVar[dict[str, str] | None] = None
     ambiguities: dict[str, frozenset[str]] | None = None
     colors: dataclasses.InitVar[dict[str, str] | None] = None
-    pairing_rules: dict[str, dict[frozenset[str], bool]] | None = None
+    pairing_rules: dict[frozenset[str], bool] | None = None
     mw_calculator: WeightCalculator | None = None
     coerce_to: OptTranslater | None = None
 
     # private attributes to be delivered via properties
     _monomers: new_alphabet.CharAlphabet = dataclasses.field(init=False)
-    _gapped: new_alphabet.CharAlphabet = dataclasses.field(init=False)
-    _gapped_missing: new_alphabet.CharAlphabet = dataclasses.field(init=False)
-    _degen: new_alphabet.CharAlphabet = dataclasses.field(init=False)
-    _degen_gapped: new_alphabet.CharAlphabet = dataclasses.field(init=False)
+    _gapped: new_alphabet.CharAlphabet | None = dataclasses.field(init=False)
+    _gapped_missing: new_alphabet.CharAlphabet | None = dataclasses.field(init=False)
+    _degen: new_alphabet.CharAlphabet | None = dataclasses.field(init=False)
+    _degen_gapped: new_alphabet.CharAlphabet | None = dataclasses.field(init=False)
     _colors: dict[str, str] = dataclasses.field(init=False)
 
     # how to connect this to the sequence constructor and avoid
@@ -445,7 +445,7 @@ class MolType:
         self,
         monomers: StrORBytes,
         make_seq: type,
-        complements: dict[str, frozenset[str]] | None,
+        complements: dict[str, str] | None,
         colors: dict[str, str] | None,
     ):
         self._colors = colors or defaultdict(_DefaultValue("black"))
@@ -508,7 +508,9 @@ class MolType:
 
         if complements:
             # assume we have a nucleic acid moltype
-            dest = "".join(complements[c] for c in self.degen_gapped_alphabet)
+            # any character not in defined complement set is defined as its
+            # own complement
+            dest = "".join(complements.get(c, c) for c in self.degen_gapped_alphabet)
             self._complement = new_alphabet.convert_alphabet(
                 self.degen_gapped_alphabet.as_bytes(),
                 dest.encode("utf8"),
@@ -1346,7 +1348,7 @@ class MolType:
             '.%s_%s{font-family: "%s",monospace !important; '
             "font-size: %dpt !important; color: %s; }"
         )
-        label = self.label or ""
+        label = self.name if self.name in _STYLE_DEFAULTS else ""
         styles = _STYLE_DEFAULTS[label].copy()
         styles.update(
             {c: f"{c}_{label}" for c in [*list(self.alphabet), "terminal_ambig"]},
diff --git a/src/cogent3/core/new_sequence.py b/src/cogent3/core/new_sequence.py
index b3029ab92..d8a643903 100644
--- a/src/cogent3/core/new_sequence.py
+++ b/src/cogent3/core/new_sequence.py
@@ -845,9 +845,14 @@ class Sequence:
         seq = str(self)
         seq = seq if limit is None else seq[:limit]
         seqlen = len(seq)
-        if self.moltype.gaps:
-            gaps = "".join(self.moltype.gaps)
-            start_gap = re.search(f"^[{gaps}]+", "".join(seq))
+        if gaps := self.moltype.gaps:
+            non_hyphen = "".join(gaps - {"-"})
+            # make sure hyphen at end of negated character group
+            # so it's not interpreted as a character range
+            chars = f"{non_hyphen}-" if "-" in gaps else non_hyphen
+            # looking for gaps at the the start of the seq
+            start_gap = re.search(f"^[{chars}]+", "".join(seq))
+            # looking for gaps at the end of the seq
             end_gap = re.search(f"[{gaps}]+$", "".join(seq))
 
             start = 0 if start_gap is None else start_gap.end()


Test patch:
diff --git a/tests/test_core/test_new_moltype.py b/tests/test_core/test_new_moltype.py
index ced2381a2..4ea682498 100644
--- a/tests/test_core/test_new_moltype.py
+++ b/tests/test_core/test_new_moltype.py
@@ -719,3 +719,16 @@ def test_has_ambiguity_validation():
 
     with pytest.raises(TypeError):
         new_moltype.DNA.has_ambiguity(1)
+
+
+def test_custom_moltype():
+    mt = new_moltype.MolType(
+        name="dna-gapped",
+        make_seq=new_sequence.DnaSequence,
+        monomers="".join(new_moltype.IUPAC_DNA_chars),
+        ambiguities=new_moltype.IUPAC_DNA_ambiguities,
+        complements=new_moltype.IUPAC_DNA_ambiguities_complements,
+        pairing_rules=new_moltype.DNA_STANDARD_PAIRS,
+        gap=".",
+    )
+    assert "." in mt.gaps
diff --git a/tests/test_core/test_new_sequence.py b/tests/test_core/test_new_sequence.py
index 3cfa1e8a3..c14f149fb 100644
--- a/tests/test_core/test_new_sequence.py
+++ b/tests/test_core/test_new_sequence.py
@@ -3089,3 +3089,17 @@ def test_to_html_bytes():
     seq = cogent3.make_seq("AGTACACTGGT", moltype="bytes", new_type=True)
     html = seq.to_html()  # should not fail
     assert isinstance(html, str)
+
+
+def test_to_html_custom_moltype():
+    mt = new_moltype.MolType(
+        name="dna-gapped",
+        make_seq=new_sequence.DnaSequence,
+        monomers="".join(new_moltype.IUPAC_DNA_chars),
+        ambiguities=new_moltype.IUPAC_DNA_ambiguities,
+        complements=new_moltype.IUPAC_DNA_ambiguities_complements,
+        pairing_rules=new_moltype.DNA_STANDARD_PAIRS,
+        gap=".",
+    )
+    seq = mt.make_seq(seq="ACG.")
+    assert isinstance(seq.to_html(), str)