Repository: rucio/rucio

Issue:
Undertaker crash when a DID is attach/detached many times
Motivation
----------
The undertaker crash when a DID is attach/detached many times. The problem is due to the archival in the history table : https://github.com/rucio/rucio/blame/master/lib/rucio/core/did.py#L714 .


Discussion:
Just on a side-note: The way we manage the PKs for history tables in the models.py is wrong.
For Oracle we do not create PKs, only in the models.py we create them, but as new experiments create their tables with the models.py install script, it should be handled differently.
I think it's historical because sqlalchemy enforces the definition of PK on table, otherwise it fails. This explains why PKs are defined in models but not in Oracle.

Yes, sqlalchemy expects a primary Key, otherwise the whole mapping from rows to addressable objects does not make sense.
What is a little bit better is to define a fake PK, like in the replication rule history table. With a history_id as PK (Which does not get created in Oracle) - but this is still kind of wrong for experiments which create their schema from models.py, because they will still end up with a PK. - But thats more of a long-term change.
In this specific case, the merge just doesnt work because merging on a table without a PK doesnt make much sense.
yes in that case the merge is wrong.
Fixing this with a PR now, more for the other tables in #1818 

Resolving patch:
diff --git a/lib/rucio/core/did.py b/lib/rucio/core/did.py
--- a/lib/rucio/core/did.py
+++ b/lib/rucio/core/did.py
@@ -14,7 +14,7 @@
 #
 # Authors:
 # - Vincent Garonne <vgaronne@gmail.com>, 2013-2018
-# - Martin Barisits <martin.barisits@cern.ch>, 2013-2017
+# - Martin Barisits <martin.barisits@cern.ch>, 2013-2018
 # - Cedric Serfon <cedric.serfon@cern.ch>, 2013-2018
 # - Ralph Vigne <ralph.vigne@cern.ch>, 2013
 # - Mario Lassnig <mario.lassnig@cern.ch>, 2013-2018
@@ -728,7 +728,6 @@ def detach_dids(scope, name, dids, session=None):
                                                              created_at=associ_did.created_at,
                                                              updated_at=associ_did.updated_at,
                                                              deleted_at=datetime.utcnow())
-        new_detach = session.merge(new_detach)
         new_detach.save(session=session, flush=False)
 
         # Send message for AMI. To be removed in the future when they use the DETACH messages
@@ -864,9 +863,11 @@ def list_content_history(scope, name, session=None):
     :param session: The database session in use.
     """
     try:
+        # query = session.query(models.DataIdentifierAssociationHistory).\
+        #    with_hint(models.DataIdentifierAssociationHistory,
+        #              "INDEX(CONTENTS_HISTORY CONTENTS_HIST_PK)", 'oracle').\
+        #    filter_by(scope=scope, name=name)
         query = session.query(models.DataIdentifierAssociationHistory).\
-            with_hint(models.DataIdentifierAssociationHistory,
-                      "INDEX(CONTENTS_HISTORY CONTENTS_HIST_PK)", 'oracle').\
             filter_by(scope=scope, name=name)
         for tmp_did in query.yield_per(5):
             yield {'scope': tmp_did.child_scope, 'name': tmp_did.child_name,
diff --git a/lib/rucio/db/sqla/models.py b/lib/rucio/db/sqla/models.py
--- a/lib/rucio/db/sqla/models.py
+++ b/lib/rucio/db/sqla/models.py
@@ -579,8 +579,11 @@ class DataIdentifierAssociationHistory(BASE, ModelBase):
     rule_evaluation = Column(Boolean(name='CONTENTS_HIST_RULE_EVAL_CHK'))
     did_created_at = Column(DateTime)
     deleted_at = Column(DateTime)
-    _table_args = (PrimaryKeyConstraint('scope', 'name', 'child_scope', 'child_name', name='CONTENTS_HIST_PK'),
-                   CheckConstraint('DID_TYPE IS NOT NULL', name='CONTENTS_HIST_DID_TYPE_NN'),
+    __mapper_args__ = {
+        'primary_key': [scope, name, child_scope, child_name]
+    }
+    # _table_args = (PrimaryKeyConstraint('scope', 'name', 'child_scope', 'child_name', name='CONTENTS_HIST_PK'),
+    _table_args = (CheckConstraint('DID_TYPE IS NOT NULL', name='CONTENTS_HIST_DID_TYPE_NN'),
                    CheckConstraint('CHILD_TYPE IS NOT NULL', name='CONTENTS_HIST_CHILD_TYPE_NN'),
                    Index('CONTENTS_HISTORY_IDX', 'scope', 'name'))
 


Test patch: