Repository: rucio/rucio

Issue:
Primary Key for replicas table is in wrong order in models.py
Motivation
----------
https://github.com/rucio/rucio/blob/a7b6fef66687f4a673b705eb738dd4d88113edac/lib/rucio/db/sqla/models.py#L736
The PK for the replicas table is in `order rse_id, scope, name` while it actually should be in order `scope, name, rse_id`
This needs to be changed and we need a proper upgrade script so this can be adapted at all installations.


Discussion:
This is potentially tricky. Changing the primary key is easily possible, however, this won't move the data already on the tablespace, it will just check that the new primary key constraint is still valid. What we actually want is for the data to be moved. So this will need an intermediate tmp_replicas table and thus some downtime as the DBA manually shovels data around. Changing the primary key can be done with the alembic upgrades, but shoveling the data can only be done manually if we don't want tmp_replicas to pollute our models.py
We can do the primary key change anyway though. And we could provide a script for all databases to do it offline. Then it's the instance admins choice when/how they apply it (if they want to do it)
It is fine to do it offline, but we need to document it somehow. I am surprised that this never showed up, but in smaller installations the queries will just be quick enough without going against the PK. And from a uniqueness point of view this PK is correct as well.

Resolving patch:
diff --git a/lib/rucio/db/sqla/migrate_repo/versions/3345511706b8_replicas_table_pk_definition_is_in_.py b/lib/rucio/db/sqla/migrate_repo/versions/3345511706b8_replicas_table_pk_definition_is_in_.py
new file mode 100644
--- /dev/null
+++ b/lib/rucio/db/sqla/migrate_repo/versions/3345511706b8_replicas_table_pk_definition_is_in_.py
@@ -0,0 +1,48 @@
+# Copyright 2013-2019 CERN for the benefit of the ATLAS collaboration.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Authors:
+# - Martin Barisits <martin.barisits@cern.ch>, 2019
+#
+# Topic: Replicas table PK definition is in wrong order
+# Revision ID: 3345511706b8
+# Revises: 9eb936a81eb1
+# Creation Date: 2019-01-30 14:20:35.058889
+
+from alembic.op import (create_primary_key, drop_constraint)
+
+from alembic import context
+
+
+# revision identifiers used by alembic
+revision = '3345511706b8'       # pylint: disable=invalid-name
+down_revision = 'bf3baa1c1474'  # pylint: disable=invalid-name
+
+
+def upgrade():
+    '''
+    Upgrade the database to this revision
+    '''
+    if context.get_context().dialect.name != 'sqlite':
+        drop_constraint('REPLICAS_PK', 'replicas')
+        create_primary_key('REPLICAS_PK', 'replicas', ['scope', 'name', 'rse_id'])
+
+
+def downgrade():
+    '''
+    Downgrade the database to the previous revision
+    '''
+    if context.get_context().dialect.name != 'sqlite':
+        drop_constraint('REPLICAS_PK', 'replicas')
+        create_primary_key('REPLICAS_PK', 'replicas', ['rse_id', 'scope', 'name'])
diff --git a/lib/rucio/db/sqla/models.py b/lib/rucio/db/sqla/models.py
--- a/lib/rucio/db/sqla/models.py
+++ b/lib/rucio/db/sqla/models.py
@@ -751,7 +751,7 @@ class RSEFileAssociation(BASE, ModelBase):
     accessed_at = Column(DateTime)
     tombstone = Column(DateTime)
     rse = relationship("RSE", backref=backref('replicas', order_by="RSE.id"))
-    _table_args = (PrimaryKeyConstraint('rse_id', 'scope', 'name', name='REPLICAS_PK'),
+    _table_args = (PrimaryKeyConstraint('scope', 'name', 'rse_id', name='REPLICAS_PK'),
                    ForeignKeyConstraint(['scope', 'name'], ['dids.scope', 'dids.name'], name='REPLICAS_LFN_FK'),
                    ForeignKeyConstraint(['rse_id'], ['rses.id'], name='REPLICAS_RSE_ID_FK'),
                    CheckConstraint('STATE IS NOT NULL', name='REPLICAS_STATE_NN'),


Test patch: