Repository: biopython/biopython

Issue:
Cannot change IDs for Residue
### Setup

I am reporting a problem with Biopython version, Python version, and operating
system as follows:

```python
import sys; print(sys.version)
import platform; print(platform.python_implementation()); print(platform.platform())
import Bio; print(Bio.__version__)
```

[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
CPython
Darwin-17.3.0-x86_64-i386-64bit
1.70

### Expected behaviour

`Residue.id(value) or res.id = value`
changes the id of the residue. This is used for renumbering a PDB.
This used to work fine for many years and now results in an error, with no obvious override mechanism. 

### Actual behaviour

```
ValueError: Cannot change id from `(' ', 8, ' ')` to `(' ', 9, ' ')`. The id `(' ', 9, ' ')` is already used for a sibling of this entity.
```

In addition, using `detach_child()` on the residue for the 'already in use sibling' does not work and results in this error:

```
  File "/Library/Python/2.7/site-packages/Bio/PDB/Entity.py", line 120, in detach_child
    child = self.child_dict[id]
KeyError: (' ', 9, ' ')
```

### Steps to reproduce

Change the id of a residue to another residue number, that is a 'child'


Discussion:
Can you make a self contained example, and/or confirm an older version of Biopython where this worked?

CC: @etal @JoaoRodrigues @lennax 
Version 1.68 works fine.  I'll get a self-contained snippet for you today @peterjc 
Using one of the ``Tests/PDB/`` files would probably be easiest for us - it might even end up as a new unit test?
Hi @jadolfbr , any luck with the snippet?

I would wager this is because of the "propagation" of parent ids, but hard to look at without a specific case.
Hey @JoaoRodrigues - Thanks for the push. I'm still cloning, but here is a quick tested snippet you should be able to run on 1A80, which is in Test/PDB. (`res.id` can also be` res.value()` to set the tupple, and both error the same way.)

```
import Bio
parser = Bio.PDB.PDBParser()
pdb = parser.get_structure("1A80", "1A80.pdb")
res = pdb[0]['L'][( ' ', 152, ' ')]
res.id = (' ', 153, ' ')
```

Now that #1567 has been merged, does this problem still occur in the latest master branch?
@peterjc - Sorry this took me so long.  I just cloned master and installed via pip.  Same issue persists.  
```
2.7.10 (default, Jul 15 2017, 17:16:57) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]

CPython
Darwin-17.3.0-x86_64-i386-64bit

1.71.dev0
```
Thanks - sadly bad news. Back to @JoaoRodrigues to investigate...
Will have a look. Thanks for the report!

A quinta, 17/05/2018, 07:55, Peter Cock <notifications@github.com> escreveu:

> Thanks - sadly bad news. Back to @JoaoRodrigues
> <https://github.com/JoaoRodrigues> to investigate...
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/biopython/biopython/issues/1551#issuecomment-389896010>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAOfEPXulVKX4jw3m4a-WZZ1oej30ywxks5tzY91gaJpZM4Sg6WQ>
> .
>

Hi.
I authored the change which causes your issue.
For your example, 1A8O, I guess an error is the expected result, since you are trying to change the id of ASP 152 to 153, but residue 153 is already present as ILE and ids should be unique.

Previous versions of biopython would allow you to change the id of the residue, but leave the chain in an inconsistent state, which is why I changed the code to raise an error.

```
pdb = parser.get_structure("1A8O", "Tests/PDB/1A8O.pdb")
res = pdb[0]['A'][(' ', 152, ' ')]
res.id=(' ', 1, ' ') # Works fine for me
res.id=(' ', 153, ' ') # Causes an Error
```
On my machine, the following works:
```
pdb = parser.get_structure("1A8O", "Tests/PDB/1A8O.pdb")
pdb[0]['A'][(' ', 153, ' ')].id=(' ', 1, ' ' )
pdb[0]['A'][(' ', 152, ' ')].id=(' ', 153, ' ' )
```

And in contrast to your report, `detach_child works` for me:

```
pdb = parser.get_structure("1A8O", "Tests/PDB/1A8O.pdb")
pdb[0]['A'].detach_child((' ', 153, ' ' ))
pdb[0]['A'][(' ', 152, ' ')].id=(' ', 153, ' ' )
```

Could you find a minimal example, where detach_child does not work?
This has broken many, many scripts out in the wild for PDB manipulation.  The chain at the end of the manipulation is consistent.  Raising an error before a whole work up of manipulation makes the manipulations harder to do, but if detach_child works, that would be great.  Perhaps it fails for consistent detaches?  I'll try the detach_child function on master. Thanks for the info.
I believe that many scripts out in the wild used the old behaviour and made sure themselves that the chain was consistent at the end of all manipulations. I had I script like that as well.
However, I felt it would be better to let biopython ensure that the chain is consistent at all times. This way, I guess, many scripts in the future will be less error prone. I had many bugs where I changed a residueA.id to `newid`, but `chain[newid]` gave a different residue, because I forgot to update the `child_dict` of the chain.

Maybe we should add a function to biopython that renumbers the whole chain. Could you give more detail, what kind of renumbering you like to achieve? 
@Bernhard10 - I'd like to modify the residue numbers and chain IDs for a PDB.  IE - this particular work is for antibody renumbering, which requires a specific numbering scheme for subsequent work (Chothia, Aho, etc.). https://www.bioc.uzh.ch/plueckthun/antibody/Numbering/NumFrame.html

PyIgClassify uses Biopython under the hood to do the renumbering - but we are now stuck on biopython 1.89.  http://dunbrack2.fccc.edu/PyIgClassify/

If there is a work-around, I would love to know.  Right now, I'm looking at rolling my own, but I'd rather not if I can avoid it. 
Would downgrading the new exception to a warning be a sensible compromise?
Absolutely.
@Bernhard10 is a warning (not an exception) an easy change to make - or are things not that simple?
As far as I remember, the question is then the following: What should `chain[id]` give, if there are suddenly two children with the same id?
We could issue a warning "chain is now inconsistent" and not update the chain's child_dict, but then we cannot provide an easy repair mechanism, if this was triggered.

In my opinion, the best solution would be to provide a `renumber_chain` function, which takes a mapping `{old_id1: new_id1, old_id2: new_id2 ...}` and renumbers the whole chain.
@jadolfbr As a workaround, I guess you can also assign to `residue._id` instead of `residue.id`
A renumber chain function would work well.  I will try the workaround.
I had another workaround for this. A bit hacky but it works;

You derive your old id to new id mapping where the new numbers have a gap (here at position 95)

```python
[(((' ', 94, ' '), 'ASP'), (' ', 94, ' ', 'D')),
 (((' ', 95, ' '), 'PRO'), (' ', 96, ' ', 'P')),
 (((' ', 96, ' '), 'THR'), (' ', 97, ' ', 'T')),
 (((' ', 97, ' '), 'PHE'), (' ', 98, ' ', 'F')),
 (((' ', 98, ' '), 'GLY'), (' ', 99, ' ', 'G')),
 (((' ', 99, ' '), 'GLN'), (' ', 100, ' ', 'Q')),
 (((' ', 100, ' '), 'GLY'), (' ', 101, ' ', 'G'))]
```
Then you reverse the list 
```python
old2new.reverse(); old2new
[(((' ', 100, ' '), 'GLY'), (' ', 101, ' ', 'G')),
 (((' ', 99, ' '), 'GLN'), (' ', 100, ' ', 'Q')),
 (((' ', 98, ' '), 'GLY'), (' ', 99, ' ', 'G')),
 (((' ', 97, ' '), 'PHE'), (' ', 98, ' ', 'F')),
 (((' ', 96, ' '), 'THR'), (' ', 97, ' ', 'T')),
 (((' ', 95, ' '), 'PRO'), (' ', 96, ' ', 'P')),
 (((' ', 94, ' '), 'ASP'), (' ', 94, ' ', 'D'))]
```
And then you assign
```python
for old, new in old2new:
    structure[0][chain_id][old[0]].id = new[:3]
```

If you do it this way there are never any clashes
I am trying to use a BioPython derived script to renumber antibodies in a pdb and I still get the same error. To date is there any way to renumber antibodies with an existing scheme (imgt, chothia, etc)?
It is now 2023.  Any way we can change this to a warning in the meantime for a complete solution?  
I disagree with the warning because the original problem still remains. Changing the id for the residue to clash with another one is not a good solution because it breaks things under the hood.

This is the code for the `id` setter:
```
      @id.setter
      def id(self, value):
          """Change the id of this entity.

          This will update the child_dict of this entity's parent
          and invalidate all cached full ids involving this entity.

          @raises: ValueError
          """
          if value == self._id:
              return
          if self.parent:
              if value in self.parent.child_dict:
                  raise ValueError(
                      f"Cannot change id from `{self._id}` to `{value}`."
                      f" The id `{value}` is already used for a sibling of this entity."
                  )
              del self.parent.child_dict[self._id]
              self.parent.child_dict[value] = self

          self._id = value
          self._reset_full_id()
```

As you see, changing `Residue.id` will cause a change in `Residue.parent.child_dict`, which is a dictionary, which requires unique keys. If you allow changing ids without this check, you end up with a mismatch between that child_dict and the actual residue.id. If you're a responsible user, you'll update them later, although that's a ton of extra work and you'll have to know what to do. For more casual users, it's a trap because you have inconsistencies..

My solution here would be to create a new empty chain and add the residues from the old chain, renumbering them as you go. Why would this not work for you?

Specifically:
```
from Bio.PDB.Chain import Chain
from Bio.PDB import PDBParser
from Bio.PDB import PDBIO

parser = PDBParser()
pdb = parser.get_structure("1A8O", "1A8O.pdb")

# Renumber chain A
chain = pdb[0]["A"]
chain.parent.detach_child(chain.id)

new_chain = Chain("A")
for res in chain:
    res = res.copy()
    h, num, ins = res.id
    res.id = (h, num + 1, ins)
    new_chain.add(res)

pdb[0].add(new_chain)

io = PDBIO()
io.set_structure(pdb)
io.save("1A8O_renum.pdb")
```
Hey guys,

It's now been 6 years and I can't tell you the number of times myself, my colleagues, and my collaborators have hit this issue - numbering is something that needs to be easily able to change.  Numbering should not have anything to do with chemical connectivity.  In protein design, numbering changes happen often and need to be taken care of.  

So, with yet another colleague coming to me who spent hours trying to debug something that shouldn't have had to use his time in the first place - can we PLEASE revert this change and bring back the old behavior.  There is much general interest in doing that in this thread, and beyond.  
Thank you.

-Jared
Hi @jadolfbr, happy to discuss it, but I did my best to provide an explanation last time you commented, provided a workaround, and there was no follow-up. I understand the frustration, but there are good arguments against reverting the behavior so I'd like to have an actual discussion about this before considering so.
I just spent three hours tracking this down while trying to renumber a pdb. The solution provide by @JoaoRodrigues is convoluted and not intuitive in the slightest bit. At most, this should be a warning not an exception.

My proposed workaround is to 1) create a new chain, 2) add the renumbered residues there, 3) swap this chain with the old chain in the structure. Quite far from convoluted.

I am willing to support the downgrading to a warning, but I just want to be sure that what you are trying to do isn't solvable otherwise. Again - the exception was added for good reason. Could you share a snippet or a script of what you are trying to do exactly?
I've shared snippets.  @sacombs  can share something probably as well.  

Here is one literally from a few days ago from ChatGPT when I was playing around and of course ended in the error:
```
def renumber_pdb(input_pdb_file, output_pdb_file):
    parser = PDB.PDBParser()
    io = PDB.PDBIO()
    structure = parser.get_structure("structure", input_pdb_file)

    # Initialize the residue number
    res_num = 1

    for model in structure:
        for chain in model:
            # Change all chains to 'A'
            chain.id = 'A'
            for residue in chain:
                # Ensure only standard residues are renumbered
                if PDB.is_aa(residue, standard=True):
                    # Renumber the residue
                    residue.id = (' ', res_num, ' ')
                    res_num += 1

    # Save the modified structure
    io.set_structure(structure)
    io.save(output_pdb_file)
```


I would like to downgrade this to a warning.  If you are already doing renumbering, I think a warning is sufficient.  Kinda like if I open the hood to my car to mess with it, going in I know that there are things there I can mess up - but the hood isn't locked down.

Right now it is.  The workaround is basically building a whole new object up instead of just modifying what we have.  Anyone who is doing this knows what they are doing and after a full numbering manipulation the state is ok.  
Here's the problem with what you want to change. The code, as is, will replace the key in `Entity.child_dict` to point to the new value. Say you have a structure with 5 residues, numbered 1 to 5. Now say you renumber 1 to 2, so you have `2, 2, 3, 4, 5`. The `child_list` of the Structure will still have 5 elements, but the `child_dict`, which is used [_everywhere_ ](https://github.com/search?q=repo%3Abiopython%2Fbiopython%20child_dict&type=code) in the code base as a proxy for an iterator for elements of an Entity, will have 4 elements. Unless you do the renumbering operation in one pass **and** affect all residues **and** there are absolutely no errors, you're OK. If anything goes wrong, there are parts of the code that will start to break. Disordered entities for instance. 

This also opens the door for issues when you have typos in your renumbering scheme. For example, this very silly example is now forbidden (for good reason) but not if we downgrade:

```
for res in struct.get_residues():
    het, num, alt = res.id
    res.id = (het, 1, alt)
```

Do you see why we (or me) are so reluctant to downgrade this? Is doing the new chain that much of a big deal in your case? The ChatGPT snippet doesn't tell me what you are trying to actually do...


Resolving patch:
diff --git a/Bio/PDB/Entity.py b/Bio/PDB/Entity.py
--- a/Bio/PDB/Entity.py
+++ b/Bio/PDB/Entity.py
@@ -9,11 +9,14 @@
 It is a simple container class, with list and dictionary like properties.
 """
 
+import warnings
+
 from collections import deque
 from copy import copy
 
 import numpy as np
 
+from Bio import BiopythonWarning
 from Bio.PDB.PDBExceptions import PDBConstructionException
 
 
@@ -173,9 +176,12 @@ def id(self, value):
             return
         if self.parent:
             if value in self.parent.child_dict:
-                raise ValueError(
-                    f"Cannot change id from `{self._id}` to `{value}`."
-                    f" The id `{value}` is already used for a sibling of this entity."
+                # See issue 1551 for details on the downgrade.
+                warnings.warn(
+                    f"The id `{value}` is already used for a sibling of this entity. "
+                    f"Changing id from `{self._id}` to `{value}` might create access "
+                    "inconsistencies to children of the parent entity.",
+                    BiopythonWarning,
                 )
             del self.parent.child_dict[self._id]
             self.parent.child_dict[value] = self


Test patch:
diff --git a/Tests/test_PDB_SMCRA.py b/Tests/test_PDB_SMCRA.py
--- a/Tests/test_PDB_SMCRA.py
+++ b/Tests/test_PDB_SMCRA.py
@@ -32,6 +32,7 @@
         "Install NumPy if you want to use Bio.PDB."
     ) from None
 
+from Bio import BiopythonWarning
 from Bio.PDB import PDBParser
 from Bio.PDB.PDBExceptions import PDBConstructionWarning
 from Bio.PDB import rotmat, Vector
@@ -322,14 +323,14 @@ def test_change_model_id(self):
         self.assertIn(2, self.structure)
         self.assertNotIn(0, self.structure)
 
-    def test_change_model_id_raises(self):
-        """Cannot change id to a value already in use by another child."""
+    def test_change_model_id_warns(self):
+        """Warning when changing id to a value already in use by another child."""
         model = next(iter(self.structure))
-        with self.assertRaises(ValueError):
+        with self.assertWarns(BiopythonWarning):
             model.id = 1
-        # Make sure nothing was changed
-        self.assertEqual(model.id, 0)
-        self.assertIn(0, self.structure)
+        # make sure children were not overwritten
+        self.assertEqual(model.id, 1)
+        self.assertEqual(len(self.structure.child_list), 2)
         self.assertIn(1, self.structure)
 
     def test_change_chain_id(self):