Repository: biopython/biopython

Issue:
Loosen restriction of FormattedSeq._table in Restriction module
Hi @peterjc, nowadays the restriction module only allows IUPAC DNA characters (see below).

https://github.com/biopython/biopython/blob/186d3825c4b6b289c347c6ea95a89ffb2717e848/Bio/Restriction/Restriction.py#L119-L125

However, even if only IUPAC characters are allowed, degenerate base codes are not interpreted as such (e.g. `N` is not treated as "any of `ACGT`" but as an `N`, character, so the below snippet returns an empty list without matches.

```python
from Bio.Restriction import EcoRI
from Bio.Seq import Seq

print(EcoRI.search(Seq("NNNNNNNNNNN")))
```

Given this, I think it would be safe to remove this restriction and allow all letters. Below some use-cases where this would allow better usage of biopython. Will submit a PR on this.

## Impact on generic use-case

This is a bit problematic for a biopython general use-case if people are using dU bases in DNA. This is done for [USER cloning](https://www.neb.com/en-gb/applications/cloning-and-synthetic-biology/user-cloning).

## Impact on pydna

In addition, in [pydna](https://github.com/pydna-group/pydna) we are thinking to use non-DNA characters to represent single-strand DNA bits in sequences. For instance, the below DNA molecule 

```
GATCGUAAAC
    CAUTUGCTAG   

PEXIGULAOCQFZJ
```

can be represented as a single string  `PEXIGULAOCQFZJ` where `PEXI` represents `ACGT` on top strand only, and `QFZJ` represents `CTAG` on bottom strand only.

The problem is that these cannot be used for restriction using the `search` method of restriction enzymes because they have non-allowed characters.

Discussion:
@manulera: cc: @BjornFJohansson @dgruano @hiyama341@peterjc: This module is without a dedicated maintainer and while there have been a few efforts to rewrite it (avoiding the current structure with a class per enzyme), that is unlikely to happen soon.

I'm unclear how relaxing this check would help, given ambiguous based as not interpreted as such anyway - nor would your RNA extensions be understood.@BjornFJohansson: @peterjc @manulera We have a use case in Pydna where we expanded the alphabet to deal with uracil as used in [USER-cloning](https://www.neb.com/en/applications/cloning-and-synthetic-biology/user-cloning?srsltid=AfmBOoq4Fy0ZgLxUrYHjo_uTOo3TZXk3pIvcAjrTu5htGo6l1zTXgKpX) for example.

The problem is not so much getting a result for non-standard bases, I think that part is fine the way it is currently. 

The problem is that the Bio.Restriction module cast all sequences as FormattedSeq where there is a [check](https://github.com/biopython/biopython/blob/186d3825c4b6b289c347c6ea95a89ffb2717e848/Bio/Restriction/Restriction.py#L156C17-L156C79) that will not allow initiating a FormattedSeq with anything else than what is in the [matching](https://github.com/biopython/biopython/blob/186d3825c4b6b289c347c6ea95a89ffb2717e848/Bio/Restriction/Restriction.py#L98)  dict defined in line 98 in the Bio.Restriction module. 

I think this check is not needed. IMHO, removing it would comply with the idea of the [responsible user](https://docs.python-guide.org/writing/style/#we-are-all-responsible-users).

That said, I think there are other options that would work as well. For example, the hard coded [_table](https://github.com/biopython/biopython/blob/186d3825c4b6b289c347c6ea95a89ffb2717e848/Bio/Restriction/Restriction.py#L142) variable could be made a default argument of `FormattedSeq.__init__`.@manulera: Hi @peterjc I think the TLDR is:

* As it is, restricting allowed characters to IUPAC `ABCDGHKMNRSTVWY` is arbitrary, since they are interpreted as the characters themselves and not as degenerate nucleotides.
* Therefore, removing this restriction and allowing other letters should not produce unintended behaviour.
* We have important use-cases where we want to pass sequences that have extra characters to the `.search` method on enzymes. Removing this constraint would enable that use-case

@peterjc: It is reassuring to have multiple voices in support when I feel out of my depth on an area of Biopython - so #4941 approved in principle, just a suggestion on the code comment following the dicussion here.@BjornFJohansson: Thank you @peterjc and @manulera. This will remove some unnecessary code in Pydna.@manulera: Also @peterjc  I am relatively familiar with the Restriction module by now. I don't have time for a full re-write (although I would love it to happen), but if you have some issues about the module feel free to tag me and I can try to help answering questions or fixing bugs if there are any.@peterjc: Thank you @manulera - I'll try to remember that next time something is reported, or if we see renewed interest in a rewrite of the digest code.@peterjc: Merged - thank you both!

Solution patch:
diff --git a/Bio/Restriction/Restriction.py b/Bio/Restriction/Restriction.py
index 684176e61..3a06a7970 100644
--- a/Bio/Restriction/Restriction.py
+++ b/Bio/Restriction/Restriction.py
@@ -119,7 +119,9 @@ DNA = Seq
 def _make_FormattedSeq_table() -> bytes:
     table = bytearray(256)
     upper_to_lower = ord("A") - ord("a")
-    for c in b"ABCDGHKMNRSTVWY":  # Only allow IUPAC letters
+    # Only allow alphabetic characters
+    # (although only ACGT are used for finding restriction sites)
+    for c in b"ABCDEFGHIJKLMNOPQRSTUVWXYZ":
         table[c] = c  # map uppercase to uppercase
         table[c - upper_to_lower] = c  # map lowercase to uppercase
     return bytes(table)


Test patch:
diff --git a/Tests/test_Restriction.py b/Tests/test_Restriction.py
index bfe009d5d..6a9020cb9 100644
--- a/Tests/test_Restriction.py
+++ b/Tests/test_Restriction.py
@@ -52,10 +52,13 @@ class SequenceTesting(unittest.TestCase):
         EcoRI.search(Seq("ATGC"))
         EcoRI.search(MutableSeq("TCAG"))
 
-    def test_non_iupac_letters(self):
-        """Test if non-IUPAC letters raise a TypeError."""
+    def test_non_allowed_characters(self):
+        """Test if non-allowed characters raise a TypeError."""
+        # Any letter is accepted, even if it's not a nucleotide
+        FormattedSeq(Seq("ABCDEFGHIJKLMNOPQRSTUVWXYZ"))
+        # Other characters are not accepted
         with self.assertRaises(TypeError):
-            seq = FormattedSeq(Seq("GATCZ"))
+            FormattedSeq(Seq("GATCZE-"))
 
     def test_formatted_seq(self):
         """Test several methods of FormattedSeq."""