Repository: rucio/rucio

Issue:
Migration of SQLAlchemy queries to new query syntax (1.4 -> 2.0)
Motivation
----------
With SQLAlchemy 2.0 the current [sqlalchemy.orm.Session](https://docs.sqlalchemy.org/en/20/orm/session_api.html#sqlalchemy.orm.Session) query syntax will become legacy and the new core query syntax should be used [sqlalchemy.sql.expression.select](https://docs.sqlalchemy.org/en/20/core/selectable.html#sqlalchemy.sql.expression.Select). This is already available in version 1.4.

Modification
------------
Over time, we need to migrate all queries to this new query syntax. This needs to be done module by module, with new PR only accepting the new query syntax. 


Discussion:
@bari12 
Which rucio modules are good ones to start with?
Most of the SQLAlchemy interaction will be in the core

https://github.com/rucio/rucio/tree/master/lib/rucio/core

Best start with the smaller ones: account, distance, identity, ...
@bari12 

@ericvaandering and I tried to generated `RemovedIn20Warning`. We enabled cms rucio int server with 
```
SetEnv PYTHONWARNINGS always::DeprecationWarning
SetEnv SQLALCHEMY_WARN_20 1
```
https://github.com/yuyiguo/containers/blob/set-python-envs/server/rucio.conf.j2#L96
https://github.com/yuyiguo/CMSKubernetes/blob/set-python-envs/docker/rucio-server/Dockerfile#L17 

I ran below commands:
```
rucio list-dataset-replicas cms:/BTag/Run2012A-v1/RAW#00e8f24c-993d-11e1-9605-003048caaace --deep
rucio list-rules "cms:/BTag/Run2012A-v1/RAW#00e8f24c-993d-11e1-9605-003048caaace"
```
In the server logs, I found:
```
k logs server-rucio-server-ff68c4d5b-7xp4d -c rucio-server | grep yuyi

[2022-04-04 23:59:54]	10.100.237.156	137.138.124.71	137.138.121.184	YkuGer33QIzLyfTp5u5@JwAAAb8	200	662	1508	43687	"GET /replicas/cms/%2FBTag%2FRun2012A-v1%2FRAW%2300e8f24c-993d-11e1-9605-003048caaace/datasets?deep=True HTTP/1.1"	"yuyi-/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=yuyi/CN=639751/CN=Yuyi Guo-unknown-243df3e747304cef9b369cf1efff8127"	"rucio-clients/1.27.2"	rucio::list-dataset-replicas

[2022-04-05 00:18:16]	10.100.237.156	137.138.124.71	137.138.121.184	YkuKyNQHC6LUJEs1VNnyZAAAAA0	200	634	1652	13665	"GET /dids/cms/%2FBTag%2FRun2012A-v1%2FRAW%2300e8f24c-993d-11e1-9605-003048caaace/rules HTTP/1.1"	"yuyi-/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=yuyi/CN=639751/CN=Yuyi Guo-unknown-243df3e747304cef9b369cf1efff8127"	"rucio-clients/1.27.2"	rucio::list-rules

```

But I did not found any `RemovedIn20Warning `. I wonder if the server was not configured to log warnings? or there are other logs I missed. BTW, there is no info in `k logs server-rucio-server-ff68c4d5b-7xp4d -c httpd-error-log  `.
I think this would rather be in the `httpd-error-log` (Or `rucio-logs` if this exists on the CMS instance); The one you show above is the access log.

But warnings I would assume it should log...
CMS does not have` rucio-logs`.  I did not find `RemovedIn20Warning` in `httpd-error-log`, but it has something like below:
```
[Mon Apr 04 16:05:04.863228 2022] [suexec:notice] [pid 15:tid 140611666167936] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Apr 04 16:05:04.887690 2022] [so:warn] [pid 15:tid 140611666167936] AH01574: module unique_id_module is already loaded, skipping
[Mon Apr 04 16:05:04.887719 2022] [so:warn] [pid 15:tid 140611666167936] AH01574: module wsgi_module is already loaded, skipping

[Mon Apr 04 16:05:04.887742 2022] [so:warn] [pid 15:tid 140611666167936] AH01574: module authn_core_module is already loaded, skipping
[Mon Apr 04 16:05:04.887752 2022] [so:warn] [pid 15:tid 140611666167936] AH01574: module cache_disk_module is already loaded, skipping
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.100.237.156. Set the 'ServerName' directive globally to suppress this message
[Mon Apr 04 16:05:04.891442 2022] [lbmethod_heartbeat:notice] [pid 15:tid 140611666167936] AH02282: No slotmem from mod_heartmonitor
```

Do you see any other Rucio debug/info message? The ones you show here are really only the apache specific ones, but not generated by Rucio. 
No, I did not see any Rucio info.
Is it possible STDERR from Rucio is going somewhere else?
I created a pull request for db/sqla to migrate to SQLAlchemy 2.0 syntax [rucio#5853](https://github.com/rucio/rucio/pull/5853) .
This is the first time I have created a PR in Rucio development.  I am unsure if the PR should be reviewed first, then I'll fix the existing conflicts and follow up with the reviewers' comments simultaneously.  Please advise. 

Resolving patch:
diff --git a/lib/rucio/core/rule.py b/lib/rucio/core/rule.py
--- a/lib/rucio/core/rule.py
+++ b/lib/rucio/core/rule.py
@@ -24,6 +24,7 @@
 from re import match
 from string import Template
 from typing import Dict, Any, Optional
+from os import path
 
 from dogpile.cache.api import NO_VALUE
 
@@ -67,6 +68,7 @@
 
 if TYPE_CHECKING:
     from sqlalchemy.orm import Session
+    from typing import List, Tuple
 
 
 REGION = make_region_memcached(expiration_time=900)
@@ -2245,11 +2247,11 @@ def approve_rule(rule_id, approver=None, notify_approvers=True, *, session: "Ses
                 text = template.safe_substitute({'rule_id': str(rule.id),
                                                  'approver': approver})
                 vo = rule.account.vo
-                recipents = __create_recipents_list(rse_expression=rule.rse_expression, filter_={'vo': vo}, session=session)
-                for recipent in recipents:
+                recipients = _create_recipients_list(rse_expression=rule.rse_expression, filter_={'vo': vo}, session=session)
+                for recipient in recipients:
                     add_message(event_type='email',
                                 payload={'body': text,
-                                         'to': [recipent[0]],
+                                         'to': [recipient[0]],
                                          'subject': 'Re: [RUCIO] Request to approve replication rule %s' % (str(rule.id))},
                                 session=session)
     except NoResultFound:
@@ -2304,11 +2306,11 @@ def deny_rule(rule_id, approver=None, reason=None, *, session: "Session"):
                                                    'approver': approver,
                                                    'reason': reason})
             vo = rule.account.vo
-            recipents = __create_recipents_list(rse_expression=rule.rse_expression, filter_={'vo': vo}, session=session)
-            for recipent in recipents:
+            recipients = _create_recipients_list(rse_expression=rule.rse_expression, filter_={'vo': vo}, session=session)
+            for recipient in recipients:
                 add_message(event_type='email',
                             payload={'body': email_body,
-                                     'to': [recipent[0]],
+                                     'to': [recipient[0]],
                                      'subject': 'Re: [RUCIO] Request to approve replication rule %s' % (str(rule.id))},
                             session=session)
     except NoResultFound:
@@ -3170,7 +3172,7 @@ def __delete_lock_and_update_replica(lock, purge_replicas=False, nowait=False, *
 
 
 @transactional_session
-def __create_rule_approval_email(rule, *, session: "Session"):
+def __create_rule_approval_email(rule: "models.ReplicationRule", *, session: "Session"):
     """
     Create the rule notification email.
 
@@ -3178,11 +3180,23 @@ def __create_rule_approval_email(rule, *, session: "Session"):
     :param session:   The database session in use.
     """
 
-    with open('%s/rule_approval_request.tmpl' % config_get('common', 'mailtemplatedir'), 'r') as templatefile:
+    filepath = path.join(
+        config_get('common', 'mailtemplatedir'), 'rule_approval_request.tmpl'  # type: ignore
+    )
+    with open(filepath, 'r') as templatefile:
         template = Template(templatefile.read())
 
-    did = rucio.core.did.get_did(scope=rule.scope, name=rule.name, dynamic_depth=DIDType.FILE, session=session)
-    rses = [rep['rse_id'] for rep in rucio.core.replica.list_dataset_replicas(scope=rule.scope, name=rule.name, session=session) if rep['state'] == ReplicaState.AVAILABLE]
+    did = rucio.core.did.get_did(
+        scope=rule.scope,
+        name=rule.name,
+        dynamic_depth=DIDType.FILE,
+        session=session
+    )
+
+    reps = rucio.core.replica.list_dataset_replicas(
+        scope=rule.scope, name=rule.name, session=session
+    )
+    rses = [rep['rse_id'] for rep in reps if rep['state'] == ReplicaState.AVAILABLE]
 
     # RSE occupancy
     vo = rule.account.vo
@@ -3208,47 +3222,51 @@ def __create_rule_approval_email(rule, *, session: "Session"):
         except Exception:
             pass
 
-    # Resolve recipents:
-    recipents = __create_recipents_list(rse_expression=rule.rse_expression, filter_={'vo': vo}, session=session)
-
-    for recipent in recipents:
-        text = template.safe_substitute({'rule_id': str(rule.id),
-                                         'created_at': str(rule.created_at),
-                                         'expires_at': str(rule.expires_at),
-                                         'account': rule.account.external,
-                                         'email': get_account(account=rule.account, session=session).email,
-                                         'rse_expression': rule.rse_expression,
-                                         'comment': rule.comments,
-                                         'scope': rule.scope.external,
-                                         'name': rule.name,
-                                         'did_type': rule.did_type,
-                                         'length': '0' if did['length'] is None else str(did['length']),
-                                         'bytes': '0' if did['bytes'] is None else sizefmt(did['bytes']),
-                                         'closed': not did['open'],
-                                         'complete_rses': ', '.join(rses),
-                                         'approvers': ','.join([r[0] for r in recipents]),
-                                         'approver': recipent[1],
-                                         'target_rse': target_rse,
-                                         'free_space': free_space,
-                                         'free_space_after': free_space_after})
+    # Resolve recipients:
+    recipients = _create_recipients_list(rse_expression=rule.rse_expression, filter_={'vo': vo}, session=session)
+
+    for recipient in recipients:
+        text = template.safe_substitute(
+            {
+                'rule_id': str(rule.id),
+                'created_at': str(rule.created_at),
+                'expires_at': str(rule.expires_at),
+                'account': rule.account.external,
+                'email': get_account(account=rule.account, session=session).email,
+                'rse_expression': rule.rse_expression,
+                'comment': rule.comments,
+                'scope': rule.scope.external,
+                'name': rule.name,
+                'did_type': rule.did_type,
+                'length': '0' if did['length'] is None else str(did['length']),
+                'bytes': '0' if did['bytes'] is None else sizefmt(did['bytes']),
+                'open': did.get('open', 'Not Applicable'),
+                'complete_rses': ', '.join(rses),
+                'approvers': ','.join([r[0] for r in recipients]),
+                'approver': recipient[1],
+                'target_rse': target_rse,
+                'free_space': free_space,
+                'free_space_after': free_space_after
+            }
+        )
 
         add_message(event_type='email',
                     payload={'body': text,
-                             'to': [recipent[0]],
+                             'to': [recipient[0]],
                              'subject': '[RUCIO] Request to approve replication rule %s' % (str(rule.id))},
                     session=session)
 
 
 @transactional_session
-def __create_recipents_list(rse_expression, filter_=None, *, session: "Session"):
+def _create_recipients_list(rse_expression: str, filter_=None, *, session: "Session"):
     """
-    Create a list of recipents for a notification email based on rse_expression.
+    Create a list of recipients for a notification email based on rse_expression.
 
     :param rse_exoression:  The rse_expression.
     :param session:         The database session in use.
     """
 
-    recipents = []  # (eMail, account)
+    recipients: "List[Tuple]" = []  # (eMail, account)
 
     # APPROVERS-LIST
     # If there are accounts in the approvers-list of any of the RSEs only these should be used
@@ -3260,47 +3278,61 @@ def __create_recipents_list(rse_expression, filter_=None, *, session: "Session")
                 try:
                     email = get_account(account=account, session=session).email
                     if email:
-                        recipents.append((email, account))
+                        recipients.append((email, account))
                 except Exception:
                     pass
 
     # LOCALGROUPDISK/LOCALGROUPTAPE
-    if not recipents:
+    if not recipients:
         for rse in parse_expression(rse_expression, filter_=filter_, session=session):
             rse_attr = list_rse_attributes(rse_id=rse['id'], session=session)
             if rse_attr.get('type', '') in ('LOCALGROUPDISK', 'LOCALGROUPTAPE'):
-                accounts = session.query(models.AccountAttrAssociation.account).filter_by(key='country-%s' % rse_attr.get('country', ''),
-                                                                                          value='admin').all()
-                for account in accounts:
+
+                query = select(
+                    models.AccountAttrAssociation.account
+                ).where(
+                    models.AccountAttrAssociation.key == f'country-{rse_attr.get("country", "")}',
+                    models.AccountAttrAssociation.value == 'admin'
+                )
+
+                for account in session.execute(query).scalars().all():
                     try:
-                        email = get_account(account=account[0], session=session).email
+                        email = get_account(account=account, session=session).email
                         if email:
-                            recipents.append((email, account[0]))
+                            recipients.append((email, account))
                     except Exception:
                         pass
 
     # GROUPDISK
-    if not recipents:
+    if not recipients:
         for rse in parse_expression(rse_expression, filter_=filter_, session=session):
             rse_attr = list_rse_attributes(rse_id=rse['id'], session=session)
             if rse_attr.get('type', '') == 'GROUPDISK':
-                accounts = session.query(models.AccountAttrAssociation.account).filter_by(key='group-%s' % rse_attr.get('physgroup', ''),
-                                                                                          value='admin').all()
-                for account in accounts:
+
+                query = select(
+                    models.AccountAttrAssociation.account
+                ).where(
+                    models.AccountAttrAssociation.key == f'group-{rse_attr.get("physgroup", "")}',
+                    models.AccountAttrAssociation.value == 'admin'
+                )
+
+                for account in session.execute(query).scalars().all():
                     try:
-                        email = get_account(account=account[0], session=session).email
+                        email = get_account(account=account, session=session).email
                         if email:
-                            recipents.append((email, account[0]))
+                            recipients.append((email, account))
                     except Exception:
                         pass
 
     # DDMADMIN as default
-    if not recipents:
-        default_mail_from = config_get('core', 'default_mail_from', raise_exception=False, default=None)
+    if not recipients:
+        default_mail_from = config_get(
+            'core', 'default_mail_from', raise_exception=False, default=None
+        )
         if default_mail_from:
-            recipents = [(default_mail_from, 'ddmadmin')]
+            recipients = [(default_mail_from, 'ddmadmin')]
 
-    return list(set(recipents))
+    return list(set(recipients))
 
 
 def __progress_class(replicating_locks, total_locks):


Test patch:
diff --git a/lib/rucio/tests/test_rule.py b/lib/rucio/tests/test_rule.py
--- a/lib/rucio/tests/test_rule.py
+++ b/lib/rucio/tests/test_rule.py
@@ -31,7 +31,7 @@
 from rucio.common.schema import get_schema_value
 from rucio.common.types import InternalAccount, InternalScope
 from rucio.common.utils import generate_uuid as uuid
-from rucio.core.account import add_account_attribute, get_usage
+from rucio.core.account import add_account_attribute, get_usage, get_account
 from rucio.core.account_limit import set_local_account_limit, set_global_account_limit
 from rucio.core.did import add_did, attach_dids, set_status
 from rucio.core.lock import get_replica_locks, get_dataset_locks, successful_transfer
@@ -50,6 +50,10 @@
 from rucio.tests.common import rse_name_generator, account_name_generator, did_name_generator
 from rucio.tests.common_server import get_vo
 
+from typing import TYPE_CHECKING
+if TYPE_CHECKING:
+    from typing import Tuple
+
 LOG = getLogger(__name__)
 
 
@@ -149,6 +153,69 @@ def setup_class(request, vo, root_account, jdoe_account, rse_factory):
 @pytest.mark.usefixtures('setup_class')
 class TestCore:
 
+    def test_add_rule_to_file_ask_approval(self, vo, mock_scope, jdoe_account):
+        """ REPLICATION RULE (CORE): Add a replication rule, asking approval"""
+        rse = rse_name_generator()
+        rse_id = add_rse(rse, vo=vo)
+
+        add_rse_attribute(rse_id, "rule_approvers", jdoe_account)
+
+        files = create_files(1, mock_scope, rse_id)
+        output = add_rule(
+            dids=files,
+            account=jdoe_account,
+            copies=1,
+            rse_expression=str(rse),
+            grouping='NONE',
+            weight=None,
+            lifetime=None,
+            locked=False,
+            subscription_id=None,
+            ask_approval=True
+        )
+        assert len(output) == 1
+        assert isinstance(output[0], str)
+
+    @pytest.mark.parametrize(
+        "param_type,param_group",
+        [
+            ("LOCALGROUPDISK", ("country", "country")),
+            ("GROUPDISK", ("physgroup", "group"))
+        ]
+    )
+    def test_create_recipients_list(
+        self, param_type: str, param_group: "Tuple[str, str]",
+        vo, random_account, function_scope_prefix
+    ):
+        """
+        REPLICATION RULE (CORE):
+        Test creation of recipients list for rses of types LOCALGROUPDISK,
+        LOCALGROUPTAPE, GROUPDISK.
+        """
+
+        rstring = function_scope_prefix + param_type
+        rsegroup, accountgroup = param_group
+
+        # create rse
+        # rse must NOT have attr 'rule_approvers'
+        rse = rse_name_generator()
+        rse_id = add_rse(rse, vo=vo)
+        add_rse_attribute(rse_id, "type", param_type)
+        add_rse_attribute(rse_id, rsegroup, rstring)
+
+        # create account
+        account: InternalAccount = random_account
+        add_account_attribute(
+            account, f"{accountgroup}-{rstring}", "admin"
+        )
+        email = get_account(account).email
+
+        # import and run relevant function
+        from rucio.core.rule import _create_recipients_list
+        out = _create_recipients_list(str(rse))
+
+        assert out == [(email, account)]
+
     def test_add_rule_file_none(self, mock_scope, jdoe_account):
         """ REPLICATION RULE (CORE): Add a replication rule on a group of files, NONE Grouping"""
         files = create_files(3, mock_scope, self.rse1_id)