Repository: electricitymaps/electricitymaps-contrib

Issue:
Costa Rica page has changed and now errors
```
Traceback (most recent call last):
  File "/home/feeder/lib/fetch_data.py", line 131, in launch_parsers
    **parser_kwargs)
  File "/home/contrib/parsers/CR.py", line 178, in fetch_production
    jsf_view_state = soup.select('#javax.faces.ViewState')[0]['value']
IndexError: list index out of range
```

https://github.com/tmrowco/electricitymap-contrib/blob/86f06791f2292c8a20e383a4642ced8b16083333/parsers/CR.py#L178


Discussion:
Changing L178 to the below gets the variable again. 
```python
jsf_view_state = soup.find("input", {"name": 'javax.faces.ViewState'})['value']
```

But now it's throwing this error which I feel has happened before.
```shell
(EM-env) chris@ThinkPad:~/electricitymap$ python parsers/CR.py 
fetch_production() ->
Traceback (most recent call last):
  File "parsers/CR.py", line 249, in <module>
    pprint(fetch_production())
  File "parsers/CR.py", line 193, in fetch_production
    df = pd.read_html(response.text, match=CHARACTERISTIC_NAME, skiprows=1, index_col=0)[0]
  File "/home/chris/EM-env/lib/python3.5/site-packages/pandas/io/html.py", line 915, in read_html
    keep_default_na=keep_default_na)
  File "/home/chris/EM-env/lib/python3.5/site-packages/pandas/io/html.py", line 749, in _parse
    raise_with_traceback(retained)
  File "/home/chris/EM-env/lib/python3.5/site-packages/pandas/compat/__init__.py", line 385, in raise_with_traceback
    raise exc.with_traceback(traceback)
ValueError: No tables found matching pattern 'Angostura'
```

Edit: yup the POST isn't getting the table anymore hence the error.
Right this is what I'm sending at this point.

```python
response2 = r.post(url, headers=headers, data=data)
```

headers some of which I've manually set.
```python
{'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Host': 'apps.grupoice.com', 'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0', 'Accept-Encoding': 'gzip, deflate, br', 'Referer': 'https://apps.grupoice.com/CenceWeb/CencePosdespachoNacional.jsf', 'Connection': 'keep-alive', 'Cookie': 'oam.Flash.RENDERMAP.TOKEN=4nxq9fcw8; JSESSIONID=0000LlGMCHNs4gUrVWu0I0tlxx1:1cjolulg9'}
```

data
```python
[('formPosdespacho_SUBMIT', '1'),
 ('formPosdespacho:txtFechaInicio_input', '16/08/2018'),
 ('formPosdespacho:pickFecha', ''),
 ('formPosdespacho:j_id_1c_selection', ''),
 ('formPosdespacho:j_id_1c_scrollState', '0,0'),
 ('javax.faces.ViewState', 'vXHdw7Ci6MYtEY1bLr4PUvqpCNHXWOVTHVJFJrnSQrUSzDdh')]
```

The cookies are being handled correctly but still no table in the response. @jarek can you take a look as well?

Resolving patch:
diff --git a/parsers/CR.py b/parsers/CR.py
--- a/parsers/CR.py
+++ b/parsers/CR.py
@@ -67,6 +67,7 @@
     u'Orotina': 'unknown',
     u'Otros': 'unknown',
     u'PE Mogote': 'wind',
+    u'PE Río Naranjo': 'hydro',
     u'PEG': 'wind',
     u'Pailas': 'geothermal',
     u'Parque Solar Juanilama': 'solar',
@@ -151,7 +152,7 @@ def df_to_data(zone_key, day, df, logger):
         results.append(data)
 
     for plant in unknown_plants:
-        logger.warning('{} is not mapped to generation type'.format(plant),
+        logger.warning(u'{} is not mapped to generation type'.format(plant),
                        extra={'key': zone_key})
 
     return results
@@ -171,21 +172,19 @@ def fetch_production(zone_key='CR', session=None,
 
     # Do not use existing session as some amount of cache is taking place
     r = requests.session()
-    url = 'https://appcenter.grupoice.com/CenceWeb/CencePosdespachoNacional.jsf'
+    url = 'https://apps.grupoice.com/CenceWeb/CencePosdespachoNacional.jsf'
     response = r.get(url)
 
     soup = BeautifulSoup(response.text, 'html.parser')
-    jsf_view_state = soup.select('#javax.faces.ViewState')[0]['value']
+    jsf_view_state = soup.find("input", {"name": 'javax.faces.ViewState'})['value']
 
     data = [
-        ('formPosdespacho', 'formPosdespacho'),
         ('formPosdespacho:txtFechaInicio_input', target_datetime.format(DATE_FORMAT)),
         ('formPosdespacho:pickFecha', ''),
-        ('formPosdespacho:j_idt60_selection', ''),
-        ('formPosdespacho:j_idt60_scrollState', '0,1915'),
+        ('formPosdespacho_SUBMIT', 1),
         ('javax.faces.ViewState', jsf_view_state),
     ]
-    response = r.post(url, cookies={}, data=data)
+    response = r.post(url, data=data)
 
     # tell pandas which table to use by providing CHARACTERISTIC_NAME
     df = pd.read_html(response.text, match=CHARACTERISTIC_NAME, skiprows=1, index_col=0)[0]


Test patch: