Repository: rucio/rucio

Issue:
Separate third_party_copy activity into read and write
Motivation
----------
Separate the `third_party_copy` activity into a `third_party_copy_read` and `third_party_copy_write` activity. Also possivle is to use Read WAN for the source. Needs discussion




Discussion:
You still call new configuration third party read/write and I'm not completely sure what exactly does this mean. For XRootD it is necessary that destination endpoint supports TPC, because only pull mode is available, but for WebDAV you can either contact destination endpoint and use pull mode or it is also fine if you contact source endpoint and use push mode. Currently FTS use for WebDAV pull mode first and than automatically fallback to push (it is possible to configure FTS not to try push) ... this makes things even more interesting in case of multisource transfers and I have no idea how FTS currently deals with such TPC transfers in case of push fallback. I guess new third party read/write configuration doesn't try to cover all possible scenarios (push model).

Resolving patch:
diff --git a/lib/rucio/db/sqla/migrate_repo/versions/2b69addda658_add_columns_for_third_party_copy_read_.py b/lib/rucio/db/sqla/migrate_repo/versions/2b69addda658_add_columns_for_third_party_copy_read_.py
new file mode 100644
--- /dev/null
+++ b/lib/rucio/db/sqla/migrate_repo/versions/2b69addda658_add_columns_for_third_party_copy_read_.py
@@ -0,0 +1,49 @@
+# Copyright 2013-2019 CERN for the benefit of the ATLAS collaboration.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Authors:
+# - Cedric Serfon <cedric.serfon@cern.ch>, 2019
+
+''' Add columns for third_party_copy_read and third_party_copy_write '''
+
+import sqlalchemy as sa
+
+from alembic import context
+from alembic.op import add_column, drop_column
+
+# Alembic revision identifiers
+revision = '2b69addda658'
+down_revision = 'bc68e9946deb'
+
+
+def upgrade():
+    '''
+    Upgrade the database to this revision
+    '''
+
+    if context.get_context().dialect.name in ['oracle', 'mysql', 'postgresql']:
+        schema = context.get_context().version_table_schema if context.get_context().version_table_schema else ''
+        add_column('rse_protocols', sa.Column('third_party_copy_write', sa.Integer), schema=schema)
+        add_column('rse_protocols', sa.Column('third_party_copy_read', sa.Integer), schema=schema)
+
+
+def downgrade():
+    '''
+    Downgrade the database to the previous revision
+    '''
+
+    if context.get_context().dialect.name in ['oracle', 'mysql', 'postgresql']:
+        schema = context.get_context().version_table_schema if context.get_context().version_table_schema else ''
+        drop_column('rse_protocols', 'third_party_copy_write', schema=schema)
+        drop_column('rse_protocols', 'third_party_copy_read', schema=schema)
diff --git a/lib/rucio/db/sqla/models.py b/lib/rucio/db/sqla/models.py
--- a/lib/rucio/db/sqla/models.py
+++ b/lib/rucio/db/sqla/models.py
@@ -17,7 +17,7 @@
 # - Joaquin Bogado <jbogado@linti.unlp.edu.ar>, 2015-2019
 # - Wen Guan <wguan.icedew@gmail.com>, 2015
 # - Martin Barisits <martin.barisits@cern.ch>, 2015-2019
-# - Cedric Serfon <cedric.serfon@cern.ch>, 2016-2018
+# - Cedric Serfon <cedric.serfon@cern.ch>, 2016-2019
 # - Mario Lassnig <mario.lassnig@cern.ch>, 2017-2019
 # - asket <asket.agarwal96@gmail.com>, 2018
 # - Hannes Hansen <hannes.jakob.hansen@cern.ch>, 2019
@@ -736,6 +736,8 @@ class RSEProtocols(BASE, ModelBase):
     write_wan = Column(Integer, server_default='0')  # if no value is provided, 0 i.e. not supported is assumed as default value
     delete_wan = Column(Integer, server_default='0')  # if no value is provided, 0 i.e. not supported is assumed as default value
     third_party_copy = Column(Integer, server_default='0')  # if no value is provided, 0 i.e. not supported is assumed as default value
+    third_party_copy_read = Column(Integer, server_default='0')  # if no value is provided, 0 i.e. not supported is assumed as default value
+    third_party_copy_write = Column(Integer, server_default='0')  # if no value is provided, 0 i.e. not supported is assumed as default value
     extended_attributes = Column(String(4000), nullable=True)
     rses = relationship("RSE", backref="rse_protocols")
     _table_args = (PrimaryKeyConstraint('rse_id', 'scheme', 'hostname', 'port', name='RSE_PROTOCOL_PK'),


Test patch: