Repository: rucio/rucio

Issue:
Postgres error in conveyor-finisher
Motivation
----------
Attempts to finish transfers end with this issue:

```
Database exception.
Details: (psycopg2.ProgrammingError) operator does not exist: boolean = integer
LINE 3: WHERE rses.deleted = false AND rses.deterministic = 0 AND (r...
                                                          ^
HINT:  No operator matches the given name and argument type(s). You might need to add explicit type casts.
 [SQL: 'SELECT rses.id AS rses_id, rses.rse AS rses_rse, rses.rse_type AS rses_rse_type, rses.deterministic AS rses_deterministic, rses.volatile AS rses_volatile, rses.staging_area AS rses_staging_area, rses.city AS rses_city, rses.region_code AS rses_region_code, rses.country_name AS rses_country_name, rses.continent AS rses_continent, rses.time_zone AS rses_time_zone, rses."ISP" AS "rses_ISP", rses."ASN" AS "rses_ASN", rses.longitude AS rses_longitude, rses.latitude AS rses_latitude, rses.availability AS rses_availability, rses.updated_at AS rses_updated_at, rses.created_at AS rses_created_at, rses.deleted AS rses_deleted, rses.deleted_at AS rses_deleted_at \nFROM rses JOIN rse_attr_map ON rses.id = rse_attr_map.rse_id \nWHERE rses.deleted = false AND rses.deterministic = %(deterministic_1)s AND (rses.availability = %(availability_1)s OR rses.availability = %(availability_2)s OR rses.availability = %(availability_3)s OR rses.availability = %(availability_4)s OR rses.availability = %(availability_5)s OR rses.availability = %(availability_6)s OR rses.availability = %(availability_7)s OR rses.availability = %(availability_8)s) AND (rses.availability = %(availability_9)s OR rses.availability = %(availability_10)s OR rses.availability = %(availability_11)s OR rses.availability = %(availability_12)s OR rses.availability = %(availability_13)s OR rses.availability = %(availability_14)s OR rses.availability = %(availability_15)s OR rses.availability = %(availability_16)s) GROUP BY rses.id, rses.rse, rses.rse_type, rses.deterministic, rses.volatile, rses.staging_area, rses.city, rses.region_code, rses.country_name, rses.continent, rses.time_zone, rses."ISP", rses."ASN", rses.longitude, rses.latitude, rses.availability, rses.updated_at, rses.created_at, rses.deleted, rses.deleted_at'] [parameters: {'availability_10': 1, 'availability_11': 2, 'availability_12': 3, 'availability_13': 4, 'availability_14': 5, 'availability_15': 6, 'availability_16': 7, 'availability_8': 7, 'availability_9': 0, 'deterministic_1': 0, 'availability_2': 1, 'availability_3': 2, 'availability_1': 0, 'availability_6': 5, 'availability_7': 6, 'availability_4': 3, 'availability_5': 4}]
2018-01-17 07:38:14,882	12917	WARNING	Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/rucio/daemons/conveyor/finisher.py", line 118, in finisher
    __handle_requests(chunk, suspicious_patterns)
  File "/usr/lib/python2.7/site-packages/rucio/daemons/conveyor/finisher.py", line 196, in __handle_requests
    undeterministic_rses = __get_undeterministic_rses()
  File "/usr/lib/python2.7/site-packages/rucio/daemons/conveyor/finisher.py", line 311, in __get_undeterministic_rses
    rses_list = list_rses(filters={'deterministic': 0})
  File "/usr/lib/python2.7/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/usr/lib/python2.7/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/usr/lib/python2.7/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/usr/lib/python2.7/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/usr/lib/python2.7/site-packages/rucio/db/sqla/session.py", line 274, in new_funct
    raise DatabaseException(str(error))
DatabaseException: Database exception.
```

Modification
------------
Seems that the Postgres refuses to do an implicit cast from integer to boolean.  The query in the finisher should do this before handing it to SQLAlchemy.



Discussion:

Resolving patch:
diff --git a/lib/rucio/daemons/conveyor/finisher.py b/lib/rucio/daemons/conveyor/finisher.py
--- a/lib/rucio/daemons/conveyor/finisher.py
+++ b/lib/rucio/daemons/conveyor/finisher.py
@@ -8,7 +8,7 @@
 # Authors:
 # - Wen Guan, <wen.guan@cern.ch>, 2015-2016
 # - Mario Lassnig, <mario.lassnig@cern.ch>, 2015
-# - Vincent Garonne, <vincent.garonne@cern.ch>, 2015
+# - Vincent Garonne, <vincent.garonne@cern.ch>, 2015-2018
 # - Cedric Serfon, <cedric.serfon@cern.ch>, 2017
 # - Martin Barisits, <martin.barisits@cern.ch>, 2017
 
@@ -308,7 +308,7 @@ def __get_undeterministic_rses():
     key = 'undeterministic_rses'
     result = region.get(key)
     if type(result) is NoValue:
-        rses_list = list_rses(filters={'deterministic': 0})
+        rses_list = list_rses(filters={'deterministic': False})
         result = [rse['id'] for rse in rses_list]
         try:
             region.set(key, result)


Test patch: