Repository: spiral-project/ihatemoney

Issue:
500 error because of exchangerate.host failing
Hi,

My own instance of ihm running 5.2.0 fails with

~~~
$ curl -i https://ihm.chown.me/
HTTP/2 500 
server: nginx
date: Sun, 10 Jul 2022 01:14:46 GMT
content-type: text/html; charset=utf-8
content-length: 265
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
content-security-policy: default-src 'self'; script-src 'self' 'unsafe-inline'; object-src 'none'
strict-transport-security: max-age=31556926; includeSubDomains
referrer-policy: strict-origin-when-cross-origin
vary: Cookie
set-cookie: session=bleh; Secure; HttpOnly; Path=/; SameSite=Lax

<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
~~~

Log says

~~~
ERROR [ihatemoney.run] Exception on / [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 2077, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1525, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.10/site-packages/flask_restful/__init__.py", line 271, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/src/ihatemoney/web.py", line 276, in home
    project_form = get_project_form()
  File "/src/ihatemoney/web.py", line 271, in get_project_form
    return ProjectForm()
  File "/usr/local/lib/python3.10/site-packages/wtforms/form.py", line 208, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/src/ihatemoney/forms.py", line 153, in __init__
    for currency_name in self.currency_helper.get_currencies()
  File "/src/ihatemoney/currency_convertor.py", line 31, in get_currencies
    for rate in self.get_rates()
  File "/usr/local/lib/python3.10/site-packages/cachetools/__init__.py", line 520, in wrapper
    v = func(*args, **kwargs)
  File "/src/ihatemoney/currency_convertor.py", line 24, in get_rates
    rates = requests.get(self.api_url).json()["rates"]
KeyError: 'rates'
~~~

Code is

~~~
 14 class CurrencyConverter(object, metaclass=Singleton):                                   
 15     # Get exchange rates                                                                
 16     no_currency = "XXX"                                                                 
 17     api_url = "https://api.exchangerate.host/latest?base=USD"                           
 18                                                                                         
 19     def __init__(self):                                                                 
 20         pass                                                                            
 21                                                                                         
 22     @cached(cache=TTLCache(maxsize=1, ttl=86400))                                       
 23     def get_rates(self):                                                                
 24         rates = requests.get(self.api_url).json()["rates"]                              
 25         rates[self.no_currency] = 1.0                                                   
 26         return rates
 ~~~

Indeed the api doesn't work anymore here:

~~~
$ curl -4i "https://api.exchangerate.host/latest?base=USD"
HTTP/2 404 
date: Sun, 10 Jul 2022 01:11:54 GMT
content-type: application/json; charset=utf-8
content-length: 17
x-dns-prefetch-control: off
x-frame-options: SAMEORIGIN
strict-transport-security: max-age=15552000; includeSubDomains
x-download-options: noopen
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
x-ratelimit-limit: 2000
x-ratelimit-remaining: 1998
access-control-allow-origin: *
access-control-allow-methods: GET
access-control-allow-credentials: false
access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept, Methods
x-forwarded-for: api.exchangerate.host
cache-control: max-age=31536000
etag: W/"11-UIVUdQWNarX1D9mk06okyEMbpS8"
vary: Accept-Encoding
cf-cache-status: HIT
age: 4306
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=PpE881p5ClV8zgzUG8mp8spUa3wngqOfil1u7w%2B%2FZZZNP7GotabJxMGXseH%2FpFBsFtlaUDo%2BmAcQB0LNlXA2zhVI%2BiA33eFoOQxvpVJlaLfRfspnXDWp2NPIIjiM1%2FY2aVHQASbmkEA%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 72856d96cad34bc5-YUL
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

{"success":false}
~~~

It makes me sad that some third party service I didn't know about broke my instance. Can you please make ihm more resilient to exchangerate.host failure?

Comments:

Patch:
diff --git a/ihatemoney/currency_convertor.py b/ihatemoney/currency_convertor.py
index 881d542..1139a44 100644
--- a/ihatemoney/currency_convertor.py
+++ b/ihatemoney/currency_convertor.py
@@ -1,3 +1,6 @@
+import traceback
+import warnings
+
 from cachetools import TTLCache, cached
 import requests
 
@@ -21,7 +24,14 @@ class CurrencyConverter(object, metaclass=Singleton):
 
     @cached(cache=TTLCache(maxsize=1, ttl=86400))
     def get_rates(self):
-        rates = requests.get(self.api_url).json()["rates"]
+        try:
+            rates = requests.get(self.api_url).json()["rates"]
+        except Exception:
+            warnings.warn(
+                f"Call to {self.api_url} failed: {traceback.format_exc(limit=0).strip()}"
+            )
+            # In case of any exception, let's have an empty value
+            rates = {}
         rates[self.no_currency] = 1.0
         return rates
 


Test patch:
diff --git a/ihatemoney/tests/main_test.py b/ihatemoney/tests/main_test.py
index 5d69130..97ab82a 100644
--- a/ihatemoney/tests/main_test.py
+++ b/ihatemoney/tests/main_test.py
@@ -382,6 +382,16 @@ class TestCurrencyConverter(unittest.TestCase):
         result = self.converter.exchange_currency(100, "USD", "EUR")
         self.assertEqual(result, 80.0)
 
+    def test_failing_remote(self):
+        rates = {}
+        with patch("requests.Response.json", new=lambda _: {}), self.assertWarns(
+            UserWarning
+        ):
+            # we need a non-patched converter, but it seems that MagickMock
+            # is mocking EVERY instance of the class method. Too bad.
+            rates = CurrencyConverter.get_rates(self.converter)
+        self.assertDictEqual(rates, {CurrencyConverter.no_currency: 1})
+
 
 if __name__ == "__main__":
     unittest.main()