Repository: biopython/biopython

Issue:
test_PDB_DSSP failure with mmcif_chain_dict[chain_id] KeyError: 'E'
Greetings,

When trying to upgrade the _python-biopython_ package to version 1.79 in Debian Sid, I'm getting the following unit test error:

```
======================================================================
ERROR: test_dssp_with_mmcif_file_and_different_chain_ids (test_PDB_DSSP.DSSP_tool_test)
Test DSSP generation from MMCIF which has different label and author chain IDs.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.9/build/Tests/test_PDB_DSSP.py", line 125, in test_dssp_with_mmcif_file_and_different_chain_ids
    dssp = DSSP(model, pdbfile)
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.9/build/Bio/PDB/DSSP.py", line 476, in __init__
    chain_id = mmcif_chain_dict[chain_id]
KeyError: 'E'
```

It turns out that the _KeyError_ occurs a couple of lines before the _try: except KeyError:_ statement which, I suppose, is the object of the test.  Note that this occurs with the patch mentionned in the [issue #3433](https://github.com/biopython/biopython/issues/3433), in order to work around the issue caused by the new default file format, so there is a good chance this is a self inflicted breakage.  If need be, you can refer to the [full log on Salsa CI](https://salsa.debian.org/med-team/python-biopython/-/jobs/2061317/raw), on which I added a couple of _print_ in the loop, in an attempt to understand what happens.

Core components versions are:

```
>>> import sys; print(sys.version)
3.9.7 (default, Sep 24 2021, 09:43:00)
>>> import platform; print(platform.python_implementation()); print(platform.platform())
CPython
Linux-5.14.0-1-amd64-x86_64-with-glibc2.32
```

Current versions of _mkdssp_ and _libcifpp_ available in Sid are:

```
$ mkdssp --version
mkdssp version 4.0.0 2020-12-18
$ dpkg -l | grep libcifpp
ii  libcifpp1  1.0.1-3  amd64  Library files for libcifpp
```

Kind Regards,
Étienne.



Discussion:
I think we can close this issue as a duplicate of #3433, not yet supporting DSSP v4. But the test failure is worth knowing about.
Hi Peter,

Yes, I thought it would be better that you'd be aware of the issue.  Please feel free to sort the bug entry in the way you think is the most sensible; I was a bit unsure of the channel to use to be honest.

For information, I uploaded to Experimental with the following workaround to skip the test, in case the _mkdssp_ version is too recent:
```
--- python-biopython.orig/Tests/test_PDB_DSSP.py
+++ python-biopython/Tests/test_PDB_DSSP.py
@@ -117,8 +117,8 @@
 
     def test_dssp_with_mmcif_file_and_different_chain_ids(self):
         """Test DSSP generation from MMCIF which has different label and author chain IDs."""
-        if self.dssp_version < StrictVersion("2.2.0"):
-            self.skipTest("Test requires DSSP version 2.2.0 or greater")
+        if self.dssp_version < StrictVersion("2.2.0") or self.dssp_version >= StrictVersion("4.0.0"):
+            self.skipTest("Test requires DSSP version between 2.2.0 and 4.0.0.")
 
         pdbfile = "PDB/1A7G.cif"
         model = self.cifparser.get_structure("1A7G", pdbfile)[0]
```

As usual, Thanks for your time!

Have a nice day,  :)
Étienne.
Hey,

I just wanted to mention something I noticed about running DSSP v4.0 on mmCIF files. The old version of DSSP used to use the label_asym_id field for the chain identifiers when running on mmCIF files. But it seems the new version uses auth_asym_id. 

Overall I see this as an improvement.

But I mention this because the current DSSP parsing code detects if the input file is a mmCIF and then assumes the dssp chains are label_asym_id and maps them to auth_asym_id. See: f85e4d91b27f72849f1288040519d6a3dd6537ab

But now in v4.0 even when passing --output-format=dssp, the chain idents are already that of auth_asym_id. So I believe the current code would try to map the chains in the dssp output to auth_asym_id even though it is already that of auth_asym_id.

So it may need to detect what version of DSSP is running and disable the mapping if it is version 4.0 or greater. Otherwise it may 1) cause a mmcif_chain_dict[chain_id] KeyError and 2) falsely map across chain ident schemes. NOTE: I have not actually tried running the biopython DSSP parser on v4.0. Just bringing this up because I think its worth mentioning.

I really wish PDB would have stuck with a single convention for chain identifiers.

Attached are some example files where you can see the difference in the chain column between versions. [Here is the PDB link for the example used.](https://www.rcsb.org/structure/1NVX)

[dssp2.3_1nvx_cif.txt](https://github.com/biopython/biopython/files/7438148/dssp2.3_1nvx_cif.txt)
[dssp4_1nvx_cif.txt](https://github.com/biopython/biopython/files/7438149/dssp4_1nvx_cif.txt)

Thanks,
Christian 


Edit to tag @jgreener64 as he will probably know best how to handle this potential issue.

Thanks again !
Good to know, should we move to support the new DSSP version.

Resolving patch:
diff --git a/Bio/PDB/DSSP.py b/Bio/PDB/DSSP.py
--- a/Bio/PDB/DSSP.py
+++ b/Bio/PDB/DSSP.py
@@ -205,6 +205,11 @@
 }
 
 
+def version(version_string):
+    """Parse semantic version scheme for easy comparison."""
+    return tuple(map(int, (version_string.split("."))))
+
+
 def ss_to_index(ss):
     """Secondary structure symbol to index.
 
@@ -221,7 +226,7 @@ def ss_to_index(ss):
     assert 0
 
 
-def dssp_dict_from_pdb_file(in_file, DSSP="dssp"):
+def dssp_dict_from_pdb_file(in_file, DSSP="dssp", dssp_version="3.9.9"):
     """Create a DSSP dictionary from a PDB file.
 
     Parameters
@@ -232,6 +237,9 @@ def dssp_dict_from_pdb_file(in_file, DSSP="dssp"):
     DSSP : string
         DSSP executable (argument to subprocess)
 
+    dssp_version : string
+        Version of DSSP excutable
+
     Returns
     -------
     (out_dict, keys) : tuple
@@ -241,7 +249,7 @@ def dssp_dict_from_pdb_file(in_file, DSSP="dssp"):
 
     Examples
     --------
-    How dssp_dict_frompdb_file could be used::
+    How dssp_dict_from_pdb_file could be used::
 
         from Bio.PDB.DSSP import dssp_dict_from_pdb_file
         dssp_tuple = dssp_dict_from_pdb_file("/local-pdb/1fat.pdb")
@@ -255,8 +263,12 @@ def dssp_dict_from_pdb_file(in_file, DSSP="dssp"):
     # and calling 'dssp' will hence not work in some operating systems
     # (Debian distribution of DSSP includes a symlink for 'dssp' argument)
     try:
+        if version(dssp_version) < version("4.0.0"):
+            DSSP_cmd = [DSSP, in_file]
+        else:
+            DSSP_cmd = [DSSP, "--output-format=dssp", in_file]
         p = subprocess.Popen(
-            [DSSP, in_file],
+            DSSP_cmd,
             universal_newlines=True,
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE,
@@ -264,8 +276,12 @@ def dssp_dict_from_pdb_file(in_file, DSSP="dssp"):
     except FileNotFoundError:
         if DSSP == "mkdssp":
             raise
+        if version(dssp_version) < version("4.0.0"):
+            DSSP_cmd = ["mkdssp", in_file]
+        else:
+            DSSP_cmd = ["mkdssp", "--output-format=dssp", in_file]
         p = subprocess.Popen(
-            ["mkdssp", in_file],
+            DSSP_cmd,
             universal_newlines=True,
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE,
@@ -460,7 +476,13 @@ def __init__(self, model, in_file, dssp="dssp", acc_array="Sander", file_type=""
             # calling 'dssp' will not work in some operating systems
             # (Debian distribution of DSSP includes a symlink for 'dssp' argument)
             try:
-                dssp_dict, dssp_keys = dssp_dict_from_pdb_file(in_file, dssp)
+                version_string = subprocess.check_output(
+                    [dssp, "--version"], universal_newlines=True
+                )
+                dssp_version = re.search(r"\s*([\d.]+)", version_string).group(1)
+                dssp_dict, dssp_keys = dssp_dict_from_pdb_file(
+                    in_file, dssp, dssp_version
+                )
             except FileNotFoundError:
                 if dssp == "dssp":
                     dssp = "mkdssp"
@@ -468,7 +490,13 @@ def __init__(self, model, in_file, dssp="dssp", acc_array="Sander", file_type=""
                     dssp = "dssp"
                 else:
                     raise
-            dssp_dict, dssp_keys = dssp_dict_from_pdb_file(in_file, dssp)
+                version_string = subprocess.check_output(
+                    [dssp, "--version"], universal_newlines=True
+                )
+                dssp_version = re.search(r"\s*([\d.]+)", version_string).group(1)
+                dssp_dict, dssp_keys = dssp_dict_from_pdb_file(
+                    in_file, dssp, dssp_version
+                )
         # If the input file is a DSSP file just parse it directly:
         elif file_type == "DSSP":
             dssp_dict, dssp_keys = make_dssp_dict(in_file)
@@ -484,7 +512,7 @@ def resid2code(res_id):
         # But MMCIFParser reads in the auth_asym_id
         # Here we create a dictionary to map label_asym_id to auth_asym_id
         # using the mmCIF file
-        if file_type == "MMCIF":
+        if file_type == "MMCIF" and version(dssp_version) < version("4.0.0"):
             mmcif_dict = MMCIF2Dict(in_file)
             mmcif_chain_dict = {}
             for i, c in enumerate(mmcif_dict["_atom_site.label_asym_id"]):
@@ -497,7 +525,7 @@ def resid2code(res_id):
         # (residue, (secondary structure, accessibility)) tuples
         for key in dssp_keys:
             chain_id, res_id = key
-            if file_type == "MMCIF":
+            if file_type == "MMCIF" and version(dssp_version) < version("4.0.0"):
                 chain_id = mmcif_chain_dict[chain_id]
                 dssp_mapped_keys.append((chain_id, res_id))
             chain = model[chain_id]
@@ -637,6 +665,6 @@ def resid2code(res_id):
             dssp_map[(chain_id, res_id)] = dssp_vals
             dssp_list.append(dssp_vals)
 
-        if file_type == "MMCIF":
+        if file_type == "MMCIF" and version(dssp_version) < version("4.0.0"):
             dssp_keys = dssp_mapped_keys
         AbstractResiduePropertyMap.__init__(self, dssp_map, dssp_keys, dssp_list)


Test patch:
diff --git a/Tests/test_PDB_DSSP.py b/Tests/test_PDB_DSSP.py
--- a/Tests/test_PDB_DSSP.py
+++ b/Tests/test_PDB_DSSP.py
@@ -15,14 +15,13 @@
 
 """Unit tests for the Bio.PDB.DSSP submodule."""
 
-from distutils.version import StrictVersion
 import re
 import subprocess
 import unittest
 import warnings
 
 try:
-    import numpy
+    import numpy  # noqa: F401
 except ImportError:
     from Bio import MissingPythonDependencyError
 
@@ -31,11 +30,21 @@
     ) from None
 
 
-from Bio import MissingExternalDependencyError
 from Bio.PDB import PDBParser, MMCIFParser
 from Bio.PDB import DSSP, make_dssp_dict
 
 
+VERSION_2_2_0 = (2, 2, 0)
+
+
+def parse_dssp_version(version_string):
+    """Parse the DSSP version into a tuple from the tool output."""
+    match = re.search(r"\s*([\d.]+)", version_string)
+    if match:
+        version = match.group(1)
+    return tuple(map(int, version.split(".")))
+
+
 def will_it_float(s):  # well played, whoever this was :)
     """Convert the input into a float if it is a number.
 
@@ -53,7 +62,7 @@ class DSSP_tool_test(unittest.TestCase):
     @classmethod
     def setUpClass(cls):
 
-        cls.dssp_version = "0.0.0"
+        cls.dssp_version = (0, 0, 0)
         is_dssp_available = False
         # Check if DSSP is installed
         quiet_kwargs = {"stdout": subprocess.PIPE, "stderr": subprocess.STDOUT}
@@ -63,7 +72,7 @@ def setUpClass(cls):
                 version_string = subprocess.check_output(
                     ["dssp", "--version"], text=True
                 )
-                cls.dssp_version = re.search(r"\s*([\d.]+)", version_string).group(1)
+                cls.dssp_version = parse_dssp_version(version_string)
                 is_dssp_available = True
             except subprocess.CalledProcessError:
                 # Older versions of DSSP
@@ -74,7 +83,7 @@ def setUpClass(cls):
                 version_string = subprocess.check_output(
                     ["mkdssp", "--version"], text=True
                 )
-                cls.dssp_version = re.search(r"\s*([\d.]+)", version_string).group(1)
+                cls.dssp_version = parse_dssp_version(version_string)
                 is_dssp_available = True
             except OSError:
                 pass
@@ -91,33 +100,39 @@ def test_dssp(self):
         """Test DSSP generation from PDB."""
         pdbfile = "PDB/2BEG.pdb"
         model = self.pdbparser.get_structure("2BEG", pdbfile)[0]
-        dssp = DSSP(model, pdbfile)
+        with warnings.catch_warnings():
+            warnings.simplefilter("ignore")  # silence DSSP warnings
+            dssp = DSSP(model, pdbfile)
         self.assertEqual(len(dssp), 130)
 
     # Only run mmCIF tests if DSSP version installed supports mmcif
     def test_dssp_with_mmcif_file(self):
         """Test DSSP generation from MMCIF."""
-        if self.dssp_version < StrictVersion("2.2.0"):
+        if self.dssp_version < VERSION_2_2_0:
             self.skipTest("Test requires DSSP version 2.2.0 or greater")
 
-        pdbfile = "PDB/2BEG.cif"
-        model = self.cifparser.get_structure("2BEG", pdbfile)[0]
-        dssp = DSSP(model, pdbfile)
-        self.assertEqual(len(dssp), 130)
+        pdbfile = "PDB/4ZHL.cif"
+        with warnings.catch_warnings():
+            warnings.simplefilter("ignore")  # silence all warnings
+            model = self.cifparser.get_structure("4ZHL", pdbfile)[0]
+            dssp = DSSP(model, pdbfile)
+        self.assertEqual(len(dssp), 257)
 
     def test_dssp_with_mmcif_file_and_nonstandard_residues(self):
         """Test DSSP generation from MMCIF with non-standard residues."""
-        if self.dssp_version < StrictVersion("2.2.0"):
+        if self.dssp_version < VERSION_2_2_0:
             self.skipTest("Test requires DSSP version 2.2.0 or greater")
 
         pdbfile = "PDB/1AS5.cif"
         model = self.cifparser.get_structure("1AS5", pdbfile)[0]
-        dssp = DSSP(model, pdbfile)
+        with warnings.catch_warnings():
+            warnings.simplefilter("ignore")  # silence DSSP warnings
+            dssp = DSSP(model, pdbfile)
         self.assertEqual(len(dssp), 24)
 
     def test_dssp_with_mmcif_file_and_different_chain_ids(self):
         """Test DSSP generation from MMCIF which has different label and author chain IDs."""
-        if self.dssp_version < StrictVersion("2.2.0"):
+        if self.dssp_version < VERSION_2_2_0:
             self.skipTest("Test requires DSSP version 2.2.0 or greater")
 
         pdbfile = "PDB/1A7G.cif"