internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: electricitymaps/electricitymaps-contrib

Issue:
MISO <-> Canada interconnector
..needs to be updated as it still pointing to Montana instead of MISO
![image](https://user-images.githubusercontent.com/1655848/35481513-36ff7488-0425-11e8-91f4-b487c037240a.png)



Discussion:
There's 2 exchanges from MISO to Ontario, Montana & and Michigan. Since both are part of the same region now we should probably simplify to one. I'm assuming we can't have 2 exchanges with the same name.
OK for merging both into a single arrow. You're right that we can't have
two with the same name.

On Mon, Jan 29, 2018 at 11:10 AM, Chris <notifications@github.com> wrote:

> There's 2 exchanges from MISO to Ontario, Montana & and Michigan. Since
> both are part of the same region now we should probably simplify to one.
> I'm assuming we can't have 2 exchanges with the same name.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/tmrowco/electricitymap/issues/1031#issuecomment-361198470>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABlEKA-nDqJhG9maEkMnl1cBFhWYXm7sks5tPZkygaJpZM4RvqJ_>
> .
>


Resolving patch:
diff --git a/parsers/CA_ON.py b/parsers/CA_ON.py
--- a/parsers/CA_ON.py
+++ b/parsers/CA_ON.py
@@ -165,11 +165,8 @@ def fetch_exchange(country_code1, country_code2, session=None):
     elif sortedCountryCodes == 'CA-ON->US-NY':
         keys = ['NEW-YORK']
         direction = 1
-    elif sortedCountryCodes == 'CA-ON->US-MI':
-        keys = ['MICHIGAN']
-        direction = 1
-    elif sortedCountryCodes == 'CA-ON->US-MN':
-        keys = ['MINNESOTA']
+    elif sortedCountryCodes == 'CA-ON->US-MISO':
+        keys = ['MICHIGAN', 'MINNESOTA']
         direction = 1
     elif sortedCountryCodes == 'CA-ON->CA-QC':
         keys = filter(lambda k: k[:2] == 'PQ', exchanges.keys())


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
Add Aruba
https://en.wikipedia.org/wiki/Aruba
Data: https://www.webaruba.com/renewable-energy-dashboard/aruba
Seen here: https://www.reddit.com/r/energy/comments/7dj5sc/electricitymap_has_new_features_carbon_intensity/dtcjw8a/?context=3
Installed capacity: https://www.webaruba.com/energy-production/power-production-figures


Discussion:
Nice! I think Aruba is currently being removed from the map in the conversion/simplification to topjson https://github.com/tmrowco/electricitymap/blob/master/web/topogen.sh#L67
"Fossil fuels" value can be mapped to the "oil" category.

See: https://www.nrel.gov/docs/fy15osti/62709.pdf

In this report by NREL it says that 84% of electricity generation were from heavy fuel oil. I assume this won't have changed. Report is from March 2015.

![image](https://user-images.githubusercontent.com/25743609/35945295-03d5ccb8-0c60-11e8-98f8-536730b41bfe.png)

I'll start working on it

Resolving patch:
diff --git a/parsers/AW.py b/parsers/AW.py
new file mode 100644
--- /dev/null
+++ b/parsers/AW.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python3
+
+import arrow
+import requests
+import datetime
+
+def fetch_production(country_code='AW', session=None):
+    r = session or requests.session()
+    url = 'https://www.webaruba.com/renewable-energy-dashboard/app/rest/results.json'
+    #User agent is mandatory or services answers 404
+    headers = {'user-agent': 'electricitymap.org'}
+    response = r.get(url, headers=headers)
+    arubaJson = response.json();
+    topData = arubaJson['dashboard_top_data']
+
+    #Values currenlty used from service    
+    fossil = topData['Fossil']
+    wind = topData['Wind']
+    solar = topData['TotalSolar']
+    
+    #We're using Fossil data to get timestamp in correct time zone
+    localDateTime = datetime.datetime.strptime(fossil['timestamp'], "%Y-%m-%d %H:%M:%S.%f")
+    zoneDateTime = arrow.Arrow.fromdatetime(localDateTime, 'America/Aruba')
+    
+    data = {
+        'countryCode': country_code,
+        'datetime': zoneDateTime.datetime,
+        'production': {
+            'oil': fossil['value'],
+            'wind': wind['value'],
+            'solar': solar['value'],
+        },
+        'storage': {},
+        'source': 'webaruba.com',
+    }
+
+    return data
+
+if __name__ == '__main__':
+    print(fetch_production())


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
Add AR missing plants
feeder_electricity_1       | VMA2TG01 is missing from the AR plant mapping!
feeder_electricity_1       | NESPDI02 is missing from the AR plant mapping!


Discussion:
VMA2**TG**01 most likely belongs to a new [**natural gas** power plant in Villa Maria with three units](http://www.enernews.com/nota/309857/albanesi-y-msu-quieren-generar-mas-electricidad-en-cordoba). So TG02 and TG03 will also follow soon. TG04 could already be included, it's coming online in 2019, as far as I understand the table.

![image](https://user-images.githubusercontent.com/25743609/36630413-4234c758-1966-11e8-9343-36a5f88825ff.png)

For NESPDI, there is a [NESP**DI**01 generator](https://datos.melendez.com.ar/csvTable/min-modernizacion/Generaci%C3%B3n-El%C3%A9ctrica-Centrales-de-Generaci%C3%B3n1a23566b-99d2-4728-848d-c55576c363cd-7.html) running on **di**esel. Searching for NESPDI02 doesn't lead anywhere at the moment, but the ending "DI" suggests it is diesel as well. So we can map it to **oil**.

![image](https://user-images.githubusercontent.com/25743609/36630511-91ccb3c4-1967-11e8-8df1-74146dd14761.png)


Resolving patch:
diff --git a/parsers/AR.py b/parsers/AR.py
--- a/parsers/AR.py
+++ b/parsers/AR.py
@@ -291,6 +291,7 @@
     'NECOTV02': 'gas',
     'NECOTV03': 'gas',
     'NECOTV04': 'gas',
+    'NESPDI02': 'oil',
     'NIH1HI': 'hydro',
     'NIH4HI': 'hydro',
     'NOMODI01': 'gas',
@@ -450,6 +451,10 @@
     'VGESTG16': 'gas',
     'VGESTG18': 'gas',
     'VIALDI01': 'oil',
+    'VMA2TG01': 'gas',
+    'VMA2TG02': 'gas',
+    'VMA2TG03': 'gas',
+    'VMA2TG04': 'gas',
     'VMARTG01': 'gas',
     'VMARTG02': 'gas',
     'VMARTG03': 'gas',


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
Taiwan TW is offline
Currently, Taiwan is grey and 24-hours-history is empty as well.

- [The link ](http://data.taipower.com.tw/opendata01/apply/file/d006001/001.txt) in the [TW.py parser](https://github.com/tmrowco/electricitymap/blob/master/parsers/TW.py) seems to show data, though. 
Maybe there have been some crucial changes?

Some other TW related things that should be fixed:
- The source link on the electricitymap website for Taiwan is not shown / shown as "?".
![image](https://user-images.githubusercontent.com/25743609/36668983-6d5ee628-1af3-11e8-91a0-db8cff5b6a64.png)

- In general, the link in README.md will show 404 error and is leading nowhere. Seems like they updated/revised their website a bit?
Here is the website with the 10-min-generation mix that should be linked in README.md:
http://www.taipower.com.tw/tc/page.aspx?mid=206&cid=404&cchk=8ccc1918-8cae-4f40-a2d0-b43454f4f218

![image](https://user-images.githubusercontent.com/25743609/36669034-afe4857a-1af3-11e8-8be7-b15da8a1b58d.png)

Taiwan TW is offline
Currently, Taiwan is grey and 24-hours-history is empty as well.

- [The link ](http://data.taipower.com.tw/opendata01/apply/file/d006001/001.txt) in the [TW.py parser](https://github.com/tmrowco/electricitymap/blob/master/parsers/TW.py) seems to show data, though. 
Maybe there have been some crucial changes?

Some other TW related things that should be fixed:
- The source link on the electricitymap website for Taiwan is not shown / shown as "?".
![image](https://user-images.githubusercontent.com/25743609/36668983-6d5ee628-1af3-11e8-91a0-db8cff5b6a64.png)

- In general, the link in README.md will show 404 error and is leading nowhere. Seems like they updated/revised their website a bit?
Here is the website with the 10-min-generation mix that should be linked in README.md:
http://www.taipower.com.tw/tc/page.aspx?mid=206&cid=404&cchk=8ccc1918-8cae-4f40-a2d0-b43454f4f218

![image](https://user-images.githubusercontent.com/25743609/36669034-afe4857a-1af3-11e8-8be7-b15da8a1b58d.png)



Discussion:

Resolving patch:
diff --git a/parsers/TW.py b/parsers/TW.py
--- a/parsers/TW.py
+++ b/parsers/TW.py
@@ -5,7 +5,7 @@
 import dateutil
 
 
-def fetch_production(country_code='TW'):
+def fetch_production(country_code='TW', session=None):
     url = 'http://data.taipower.com.tw/opendata01/apply/file/d006001/001.txt'
     response = requests.get(url)
     data = response.json()


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
AU battery returning type error
```
fetch_production("AUS-SA") ->
Traceback (most recent call last):
  File "AU.py", line 558, in <module>
    print(fetch_production('AUS-SA'))
  File "AU.py", line 422, in fetch_production
    data['storage']['battery'] = AU_battery.fetch_SA_battery()
  File "/home/chris/electricitymap/parsers/lib/AU_battery.py", line 30, in fetch_SA_battery
    latest = json.loads(data[-1])
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
```


Discussion:

Resolving patch:
diff --git a/parsers/lib/AU_battery.py b/parsers/lib/AU_battery.py
--- a/parsers/lib/AU_battery.py
+++ b/parsers/lib/AU_battery.py
@@ -21,11 +21,9 @@ def fetch_SA_battery(session=None):
 
     s = session or requests.Session()
     req = s.get(nemlog_url)
-
     data = []
-    for line in req.iter_lines():
+    for line in req.iter_lines(decode_unicode=True):
         data.append(line)
-
     try:
         latest = json.loads(data[-1])
     except IndexError:


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
Exception BO
feeder_electricity_1  | Exception while fetching production of BO
feeder_electricity_1  | Traceback (most recent call last):
feeder_electricity_1  |   File "feeder_electricity.py", line 111, in fetch_production
feeder_electricity_1  |     objs = parser(country_code, session)
feeder_electricity_1  |   File "/home/feeder/lib_electricitymap/parsers/BO.py", line 149, in fetch_production
feeder_electricity_1  |     sum(datapoint['production'].values()) != 0.0]):
feeder_electricity_1  | AttributeError: 'NoneType' object has no attribute 'values'


Discussion:
Regular occurrence or one off? I think I understand why this is happening. 
![image](https://user-images.githubusercontent.com/1655848/36641315-01c73110-1a2e-11e8-86f3-d4ab36c10810.png)

We should really try to set up a parser dashboard when we have some time.
Yup #1003 would help a lot here.  All that needs to happen first (at high level) is to pipe the EM logger/error handler to another page on the electricity map domain, we can make it look nice later.
Maybe a simple first step would be to set up an ELK stack (ElasticSearch + Kibana) on which we can see logs of parsers on two levels (warnings + errors).
I don't have any experience with that sort of thing so whatever you think is easiest/quickest is fine by me. 

Resolving patch:
diff --git a/parsers/BO.py b/parsers/BO.py
--- a/parsers/BO.py
+++ b/parsers/BO.py
@@ -144,9 +144,13 @@ def fetch_production(country_code='BO', session=None):
     #    updates ~5mins after the hour so condition 2 will pass.
     valid_data = []
     for datapoint in data:
-        if all([datapoint['production'] is not None,
-                now.datetime > datapoint['datetime'],
-                sum(datapoint['production'].values()) != 0.0]):
+        if datapoint['production'] is None:
+            continue
+        elif now.datetime < datapoint['datetime']:
+            continue
+        elif sum(datapoint['production'].values()) == 0.0:
+            continue
+        else:
             valid_data.append(datapoint)
 
     return valid_data


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
AR: Plant mappings missing
March 16th 2018, 12:14:50.317 | WARNING | AR | ALUATG08 is missing from the AR plant mapping!
March 16th 2018, 12:14:50.317 | WARNING | AR | ALUATG07 is missing from the AR plant mapping!
March 16th 2018, 12:14:50.316 | WARNING | AR | ALUATV01 is missing from the AR plant mapping!



Discussion:
Aluar gas power plant. Will add these four to AR.py mapping

![image](https://user-images.githubusercontent.com/25743609/37518555-e31bae16-2915-11e8-8cea-fbc3565cb44b.png)


https://www.industryabout.com/country-territories-3/1230-argentina/fossil-fuels-energy/17340-puerto-madryn-aluar-gas-power-plant

Resolving patch:
diff --git a/parsers/AR.py b/parsers/AR.py
--- a/parsers/AR.py
+++ b/parsers/AR.py
@@ -43,6 +43,10 @@
     'ALEMDI01': 'oil',
     'ALICHI': 'hydro',
     'ALOMDI01': 'gas',
+    'ALUATG06': 'gas',
+    'ALUATG07': 'gas',
+    'ALUATG08': 'gas',
+    'ALUATV01': 'gas',
     'ALUMDI01': 'oil',
     'AMEGHI': 'hydro',
     'ANATDI01': 'gas',


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
Brazil solar data out of range
```
Traceback (most recent call last):
  File "feeder_electricity.py", line 148, in fetch_production
    objs = parser(country_code, session, logger=public_logger)
  File "/home/electricitymap/parsers/BR.py", line 131, in fetch_production
    generation = production_processor(gd, zone_key)
  File "/home/electricitymap/parsers/BR.py", line 81, in production_processor
    raise ValueError('the solar value is out of range')
ValueError: the solar value is out of range
```


Discussion:
I see `'solar': -1.1016345` being given for BR-CS right now. It looks like the parser is coded to raise ValueError on solar values `< -1`.

I guess a bigger question is how we want to handle this? -1.1 MW is a pretty obvious marginal reading that should be safe to overwrite to 0. Do we want to set limits of what's acceptable in each parser separately? That'll be a lot of edge cases to chase.
If we decide that -1 MW we can tolerate but more than that is an error, then this isn't a bug at all.
There is a function in https://github.com/tmrowco/electricitymap/blob/master/parsers/lib/validation.py#L74 that would work for handling this (anything smaller than -5 is set to None). Maybe we should move it into quality.py and apply that to all small negative numbers returned. As you say -1/-2/-3 are likely insignificant errors and not worth losing the whole datapoint over.

Resolving patch:
diff --git a/parsers/BR.py b/parsers/BR.py
--- a/parsers/BR.py
+++ b/parsers/BR.py
@@ -1,9 +1,13 @@
 #!/usr/bin/env python3
 
-import arrow
 from collections import defaultdict
+
+import arrow
 import requests
 
+from .lib.validation import validate
+
+
 url = 'http://tr.ons.org.br/Content/GetBalancoEnergetico/null'
 
 generation_mapping = {
@@ -72,14 +76,6 @@ def production_processor(json_data, zone_key):
     region = regions[zone_key]
     breakdown = json_data[region][u'geracao']
     for generation, val in breakdown.items():
-        # tolerance range
-        if -1 <= totals['solar'] < 0:
-            totals['solar'] = 0.0
-
-        # not tolerance range
-        if totals['solar'] < -1:
-            raise ValueError('the solar value is out of range')
-
         totals[generation] += val
 
     # BR_CS contains the Itaipu Dam.
@@ -140,6 +136,9 @@ def fetch_production(zone_key, session=None, target_datetime=None, logger=None):
       'source': 'ons.org.br'
     }
 
+    datapoint = validate(datapoint, logger,
+                         remove_negative=True, required=['hydro'], floor=1000)
+
     return datapoint
 
 


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
WARNING - thermal_production: PALENQUE is missing from the DO plant mapping



Discussion:
any info on the amount of MWs? could be oil or solar
It's about 30 MW.
There is an [oil power plant with 4x6MW](http://web.codemon.com/project/wartsila-dominicana/) located [here](https://www.google.de/maps/place/18%C2%B016'21.5%22N+70%C2%B008'25.6%22W/@18.2726378,-70.157954,14z/data=!3m1!4b1!4m5!3m4!1s0x0:0x0!8m2!3d18.2726389!4d-70.1404444)

Resolving patch:
diff --git a/parsers/DO.py b/parsers/DO.py
--- a/parsers/DO.py
+++ b/parsers/DO.py
@@ -64,6 +64,7 @@
                  u'MONTE PLATA SOLAR': 'solar',
                  u'MONTE RIO': 'oil',
                  u'PALAMARA': 'oil',
+                 u'PALENQUE': 'oil',
                  u'PARQUE ENERGETICO LOS MINA CC PARCIAL': 'gas',
                  u'PARQUE ENERGETICO LOS MINA CC TOTAL': 'gas',
                  u'PIMENTEL 1': 'oil',


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
Map Uruguay's "unknown" to "oil"
For Uruguay, everything that is reported as "thermal" is currently assigned to "unknown" category.

![image](https://user-images.githubusercontent.com/25743609/38982325-daec3ee2-43c1-11e8-96e9-16fb636deb55.png)

**There are some hints this is entirely based on "oil" as fuel.**

1. the IEA yearly reports show oil as only possible source for "thermal" https://www.iea.org/statistics/statisticssearch/report/?country=URUGUAY&product=electricityandheat&year=2015

2. Based on the "list of power stations" https://en.wikipedia.org/wiki/List_of_power_stations_in_Uruguay,
I found info in 3.

3. going through the list, I always ended up on http://globalenergyobservatory.org/, reporting "fuel oil" or "distillate oil" as primary fuel:
3.1 http://globalenergyobservatory.org/geoid/42286
3.2 http://globalenergyobservatory.org/geoid/42287
3.3 http://globalenergyobservatory.org/geoid/42290
3.4 http://globalenergyobservatory.org/geoid/42291

If that's enough evidence, let's change it in the parser! 😃


Discussion:

Resolving patch:
diff --git a/parsers/UY.py b/parsers/UY.py
--- a/parsers/UY.py
+++ b/parsers/UY.py
@@ -15,7 +15,7 @@
     'Eólica': 'wind',
     'Fotovoltaica': 'solar',
     'Biomasa': 'biomass',
-    'Térmica': 'unknown'
+    'Térmica': 'oil'
 }
 INV_MAP_GENERATION = dict([(v, k) for (k, v) in MAP_GENERATION.items()])
 


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
IN-AP has changed its data url
The new link is https://core.ap.gov.in/CMDashBoard/UserInterface/Power/PowerReport.aspx (same page layout I think). Old link returns 404.


Discussion:
@systemcatch yup, that page looks just like the old one. I checked all the span_id attributes in the parser, they are still identical.

Resolving patch:
diff --git a/parsers/IN_AP.py b/parsers/IN_AP.py
--- a/parsers/IN_AP.py
+++ b/parsers/IN_AP.py
@@ -3,7 +3,6 @@
 from requests import Session
 from .lib import zonekey, IN, web
 
-
 def fetch_production(zone_key='IN-AP', session=None, target_datetime=None, logger=None):
     """Fetch Andhra Pradesh  production"""
     if target_datetime:
@@ -12,7 +11,7 @@ def fetch_production(zone_key='IN-AP', session=None, target_datetime=None, logge
     zonekey.assert_zone_key(zone_key, 'IN-AP')
 
     html = web.get_response_soup(zone_key,
-                                 'http://www.core.ap.gov.in/CMDashBoard/UserInterface/Power/PowerReport.aspx', session)
+                                 'https://core.ap.gov.in/CMDashBoard/UserInterface/Power/PowerReport.aspx', session)
     india_date = IN.read_datetime_from_span_id(html, 'lblPowerStatusDate', 'DD-MM-YYYY HH:mm')
 
     hydro_value = IN.read_value_from_span_id(html, 'lblHydel')
@@ -59,7 +58,7 @@ def fetch_consumption(zone_key='IN-AP', session=None, target_datetime=None, logg
     zonekey.assert_zone_key(zone_key, 'IN-AP')
 
     html = web.get_response_soup(zone_key,
-                                 'http://www.core.ap.gov.in/CMDashBoard/UserInterface/Power/PowerReport.aspx', session)
+                                 'https://core.ap.gov.in/CMDashBoard/UserInterface/Power/PowerReport.aspx', session)
     india_date = IN.read_datetime_from_span_id(html, 'lblPowerStatusDate', 'DD-MM-YYYY HH:mm')
 
     demand_value = IN.read_value_from_span_id(html, 'lblGridDemand')


Test patch:
Repository: electricitymaps/electricitymaps-contrib

Issue:
Costa Rica has a new generation type and gives too many warnings
There seems to be a new generation type for Costa Rica that is causing a warning.

https://kibana.electricitymap.org/app/kibana#/doc/93e631f0-245f-11e8-a779-9d01de8d7a71/logstash-2018.04.28/doc?id=N-7nDGMBoL7AEh1EXs9P&_g=()

```unknown is not mapped to generation type```

However looking at the logs for the past week [here](https://kibana.electricitymap.org/app/kibana#/discover/1710fdd0-2460-11e8-a779-9d01de8d7a71?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now%2Fw,mode:quick,to:now%2Fw))&_a=(columns:!(level,extra.key,message),filters:!(('$state':(store:appState),exists:(field:level),meta:(alias:!n,disabled:!f,index:'93e631f0-245f-11e8-a779-9d01de8d7a71',key:level,negate:!f,type:exists,value:exists)),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'93e631f0-245f-11e8-a779-9d01de8d7a71',key:level,negate:!t,params:(query:INFO,type:phrase),type:phrase,value:INFO),query:(match:(level:(query:INFO,type:phrase))))),index:'93e631f0-245f-11e8-a779-9d01de8d7a71',interval:auto,query:(language:lucene,query:CR),sort:!('@timestamp',desc))) the warning happens multiple times on one run. If I remember rightly the parser gets data for each hour available, we should alter it to only warn once per run to reduce clutter.


Discussion:
Comparing the plant mapping in the parser to the table on the website, "Los Negros II" is the only missing unit I could find. It's "[a hydro power plant](http://www.carboncr.com/proyectos/costa-rica/negros-ii)" and was added to the table on 23rd and started reporting generation on 25th April.

Resolving patch:
diff --git a/parsers/CR.py b/parsers/CR.py
--- a/parsers/CR.py
+++ b/parsers/CR.py
@@ -52,6 +52,7 @@
     u'La Esperanza (CoopeL)': 'hydro',
     u'La Joya': 'hydro',
     u'Los Negros': 'hydro',
+    u'Los Negros II': 'hydro',
     u'Los Santos': 'wind',
     u'MOVASA': 'wind',
     u'Matamoros': 'unknown',
@@ -59,6 +60,7 @@
     u'Miravalles II': 'geothermal',
     u'Miravalles III': 'geothermal',
     u'Miravalles V': 'geothermal',
+    u'Moín I': 'oil',
     u'Moín II': 'oil',
     u'Moín III': 'oil',
     u'Orosí': 'wind',
@@ -133,6 +135,7 @@ def df_to_data(zone_key, day, df, logger):
     df = df.iloc[:, :-1]
 
     results = []
+    unknown_plants = set()
     hour = 0
     for column in df:
         data = empty_record(zone_key)
@@ -141,13 +144,16 @@ def df_to_data(zone_key, day, df, logger):
             source = POWER_PLANTS.get(index)
             if not source:
                 source = 'unknown'
-                logger.warning('{} is not mapped to generation type'.format(source),
-                               extra={'key': zone_key})
+                unknown_plants.add(index)
             data['datetime'] = data_time
             data['production'][source] += max(0.0, value)
         hour += 1
         results.append(data)
 
+    for plant in unknown_plants:
+        logger.warning('{} is not mapped to generation type'.format(plant),
+                       extra={'key': zone_key})
+
     return results
 
 


Test patch: