Repository: electricitymaps/electricitymaps-contrib

Issue:
Moldova MD / emission factor for gas
@brunolajoie

In #926, I mentioned that electricity production from natural gas in *Moldova* happens with an efficiency of ~22%, which leads to CO2-emissions of ~1000 g/kWh for the gas units. However, this efficiency was based on numbers from 2008, but I don't think that more efficient gas units were put into operation in the past decade in MD.

Here's the [source for the efficiency](https://www.google.de/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjhvbnshpLYAhWIEVAKHaIIACQQFgguMAA&url=http%3A%2F%2Fget-moldau.de%2Fdownload%2Fpolicypapers%2F2010%2FGET%2520Moldova_PP%252001%25202010_en.pdf&usg=AOvVaw37WEhiS1RHEn9lR52Ono--) (page -3-) 
![image](https://user-images.githubusercontent.com/25743609/34484931-345a12fe-efc8-11e7-8187-772e643b2e1c.png)

I used [this as source for the CO2 emissions](http://www.klimaneutral-handeln.de/php/kompens-berechnen.php#rechner): 

burning 1 kWh of natural gas leads to 220 g CO2 emissions.
So 220 g/kWh(thermal) / 22% = 1000 g/kWh(electrical).

If you think the data is precise, let's use a country specific factor for MD ;)


Discussion:
@alixunderplatz there is no such thing as "1 kWh of natural gas".

There are 52.16 kG of CO2 for every 1 million BTU of Methane combusted.  How the efficiency (heat rate) of the plant determines how many kWh of electricity produced from the combustion of a specific amount of natural gas releasing a given amount of CO2.  Everything else in the mass balance is lost as heat.

You can read a bit more here, but IPCC values are a very poor determiner of realtime CO2 emissions per kWH.  https://www.engineeringtoolbox.com/co2-emission-fuels-d_1085.html
@HansHyde 
My intention for opening this MD-related issue: I just wanted to point out, that for this rather overseeable country and its few generation units with a poor efficiency (info that I found by accident while searching for data for MD), we might as well change the factor here, like we did for Norwegian hydro or Estonian oil shale.


1 million BTU of pure methane = 293 kWh of methane in derived SI units.
So the converted emission factor from combusting 1 million BTU methane is 178 g(CO2)/kWh(CH4) according to your data. The site you linked says 180 g/kWh which is the same.
That is also fairly close to the 220 g/kWh and close to other info I could find (~200 g/kWh) for *natural* gas, which usually contains higher alkanes with more carbon atoms, too. So I don't get the point why there shouldn't be such thing as "1 kWh of natural gas".
@alixunderplatz 

Hi Alex, for simplicity's sake I will run 4000 MMBTU through 4 different "NG generators", and you can tell me if it is close.

4000 MMBTU over 1 hour
- GE 9A (Harriet) in its simple cycle mode will produce 480 MW/h
-  from its HRSG & Steam Turbine it will produce 230 MW/h more without additional MMBTU
- a conventional boiler with steam turbine would produce 366 MW/h, and would also require significant additional MMBTU to "ramp up" before it produced any MWHr.
- from a lower rated simple cycle gas turbine 420 MW/h

The fuel in was the same, so you could calculate the kgCO2 emitted, then compare against the g/kWh number if you like.
  
Ps your conversion above is wrong. I only know because today I used the same and energy magically appeared. When I asked others what I was doing wrong, one insisted the 1 MMBTU = 293KWh was correct.

It is not, because it forgets to take into account time.

A BTU = 1055 Joule. A Joule = a Watt-sec. Therefore, a BTU equals about 17.5 Watt-minutes.

https://en.m.wikipedia.org/wiki/British_thermal_unit
@HansHyde Hello Hans!
I totally get the point, that not every generating unit has the same efficiency, therefore different emissions appear. Some may produce more electricity from the primary energy provided by natural gas due to plant layout, load factor and operating mode (continuous or on-off-on-off), ambient temperature, cooling water temperature, gas composition and whatsoever.
By the way, what do you mean by "It is not, because it forgets to take into account time."? What time do you refer to? In terms of energy: 17.5 Watt minutes are still equal to 0.293 watt hours and 1055 watt seconds and 1 BTU.

However, in the case of **Moldova**, which this issue is all about, NG clearly provides base load:

![image](https://user-images.githubusercontent.com/25743609/34610444-69ee9a42-f221-11e7-8b6f-7bfdfdfd03cd.png)

**No ramp up during usual operation** of the gas CHPPs in the last days. The efficiency is **given as 22%.** Therefore, from the calculations above, I suggested 1000 g/kWh. This isn't equal to LCA emissions for these specific units, I'm aware of that. But for this specific country, it's the "more likely" value.

If this is of any relevance for a decision on that issue: Adding this emission factor would only influence MD itself, since currently there is no active connection with Romania and there are exlusively imports from Ukraine (for balancing). At the moment for example, emissions would increase from ~600 g/kWh to ~700 g/kWh.

I don't mind if this issue is closed without any changes to the emissions, I'll rather focus on finding more relevant stuff to cover the map :)
I have nothing to add in under 10 words.
Thanks for the analysis @alixunderplatz. Let's keep that in mind for later on. I'd rather wait a bit to see if we can't find a more "standardized" approach to country-by-country GHG intensity factor refining.  See #738  If not, we'll come back to this issue and see what we can do. 
This is not related to what I previously wrote about the smaller power plants, but suits the title of this issue:

I stumbled across some data regarding **the entire "unknown" part** of the generation for the first 9 months of 2017, which is supplied by the largest power station of MD with 2520 MW.

Actually, nearly 100% of its generation came from natural gas in these months of 2017 as well, so we could finally reassign this from "unknown" to "gas" :)

here is the mix:
gas / coal / oil
99.92 / 0 / 0.08 %

Note: the power station is capable of using oil, gas and coal as fuel, therefore we had put its capacity into the "other/unknown" category.

If we changed this, should we rather
- put the power station to "gas" instead of "unknown" or 
- assign 480 g/kWh to the "unknown" category and add the reason in the annotations/description?


[LINK TO SOURCE](http://moldgres.com/wp-content/uploads/2017/10/%D0%9F%D1%80%D0%BE%D0%B8%D0%B7%D0%B2%D0%BE%D0%B4%D1%81%D1%82%D0%B2%D0%B5%D0%BD%D0%BD%D1%8B%D0%B5-%D0%BF%D0%BE%D0%BA%D0%B0%D0%B7%D0%B0%D1%82%D0%B5%D0%BB%D0%B8-%D0%B7%D0%B0-9-%D0%BC%D0%B5%D1%81%D1%8F%D1%86%D0%B5%D0%B2-2017-%D0%B3%D0%BE%D0%B4%D0%B0.pdf)

![image](https://user-images.githubusercontent.com/25743609/34675852-1ecc08da-f48b-11e7-9fdc-e11a79899ed8.png)


@alixunderplatz is there something that needs to be changed on the map based on your analysis? Else I think we can close the issue?
@corradio 

* the "unknown" category could be assigned the gas intensity of 480 g/kWh. This would keep the nice details of ramping of the large multi-fuel power plant which we map to unknown. Based on last year's figures, that power plant generated using a mix of 99.92% gas / 0.08% oil.

![image](https://user-images.githubusercontent.com/25743609/39618891-f9401542-4f85-11e8-86fb-a6227e5edd58.png)

* the carbon intensity for the CHP gas units providing the gas baseload stripe could be increased to 1000 g/kWh based on their efficiency of 22%. I calculated this on my own, see the first comment in this issue for details. But since these figures were old and the heat output is unknown, I'd leave it as it is.

But, all in all, I don't mind leaving Moldova as it is, because it is basically importing from UA all the time and it's carbon intensity has absolutely no influence on other zones. 

So I'll close.
Thanks @alixunderplatz !
My guess is, If unknown is composed of 99.9% gas, it's a super quick win to assign it to "gas", and leave carbon intensity of gas as it is from IPCC. First its usefull to understand where your electricity comes from, and second it's gonna be usefull later on if we refined carbon intensity factors per fuel that differs per country.

@brunolajoie I agree, so I'll change it in the mapping.

![image](https://user-images.githubusercontent.com/25743609/39620375-3d5a6502-4f8b-11e8-9349-cc85c738b011.png)



Resolving patch:
diff --git a/parsers/MD.py b/parsers/MD.py
--- a/parsers/MD.py
+++ b/parsers/MD.py
@@ -4,101 +4,301 @@
 """Parser for Moldova."""
 
 import arrow
-from operator import itemgetter
 import requests
+from collections import namedtuple
 
-TYPE_MAPPING = {
-    u'tmva476': 'hydro',  # NHE Costeşti (run-of-river) #2 index
-    u'tmva112': 'hydro',  # NHE Dubăsari (run-of-river) #4 index
-    u'tmva367': 'gas',  # CET Nord (CHPP) #3 index
-    u'tmva42': 'gas',  # CET-1 Chişinău (CHPP) #6 index
-    u'tmva378': 'gas',  # CET-2 Chişinău (CHPP) #5 index
-    u'tmva1024': 'gas',  # CERS Moldovenească (fuel mix 2017 99.92% gas, 0.08% oil) #7 index
-}
+# Supports the following formats:
+# - type=csv for zip-data with semicolon-separated-values
+# - type=array for a 2D json-array containing an array for each datapoint
+# - type=html for a HTML-table (default when no type is given)
+archive_base_url = 'https://moldelectrica.md/utils/archive2.php?id=table&type=array'
+
+# Format for date and time used in archive-datapoints.
+archive_datetime_format = 'YYYY-MM-DD HH:mm'
+# Format for date only used in the archive-url.
+archive_date_format = 'DD.MM.YYYY'
+
+# Fields that can be fetched from archive_url in order.
+archive_fields = (
+    'datetime',
+    'consumption',
+    'planned_consumption',
+    'production',
+    'planned_production',
+    'tpp',  # production from thermal power plants
+    'hpp',  # production from thermal power plants
+    'res',  # production from renewable energy sources
+    'exchange_UA_to_MD',
+    'planned_exchange_UA_to_MD',
+    'exchange_RO_to_MD',
+    'planned_exchange_RO_to_MD',
+)
+
+# Datapoint in the archive-data.
+ArchiveDatapoint = namedtuple('ArchiveDatapoint', archive_fields)
 
 display_url = 'http://www.moldelectrica.md/ro/activity/system_state'
-data_url = 'http://www.moldelectrica.md/utils/load4.php'
+data_url = 'http://www.moldelectrica.md/utils/load5.php'
+
+# To match the fields with the respective index and meaning,
+# I used the following code which may be used in the future for maintenance:
+# https://gist.github.com/Impelon/09407f739cdff05134f8c77cb7a92ada
+
+# Fields that can be fetched from data_url linked to production.
+# Types of production and translations have been added manually.
+production_fields = (
+    {'index': 2, 'name': 'NHE Costeşti', 'type': 'hydro'},  # run-of-river
+    {'index': 3, 'name': 'CET Nord', 'type': 'gas'},  # CHPP
+    {'index': 4, 'name': 'NHE Dubăsari', 'type': 'hydro'},
+    {'index': 5, 'name': 'CET-2 Chişinău', 'type': 'gas'},  # CHPP
+    {'index': 6, 'name': 'CET-1 Chişinău', 'type': 'gas'},  # CHPP
+    {'index': 7, 'name': 'CERS Moldovenească', 'type': 'gas'},  # fuel mix: 99.94% gas, 0.01% coal, 0.05% oil (2020)
+)
+# Other relevant fields that can be fetched from data_url.
+other_fields = (
+    {'index': 26, 'name': 'consumption'},
+    {'index': 27, 'name': 'generation'},
+    {'index': 28, 'name': 'exchange UA->MD'},
+    {'index': 29, 'name': 'exchange RO->MD'},
+    {'index': 30, 'name': 'utility frequency'},
+)
+
+# Further information on the equipment used at CERS Moldovenească can be found at:
+# http://moldgres.com/o-predpriyatii/equipment
+# Further information on the fuel-mix used at CERS Moldovenească can be found at:
+# http://moldgres.com/search/%D0%9F%D1%80%D0%BE%D0%B8%D0%B7%D0%B2%D0%BE%D0%B4%D1%81%D1%82%D0%B2%D0%B5%D0%BD%D0%BD%D1%8B%D0%B5%20%D0%BF%D0%BE%D0%BA%D0%B0%D0%B7%D0%B0%D1%82%D0%B5%D0%BB%D0%B8
+# (by searching for 'Производственные показатели' aka. 'Performance Indicators')
+# Data for the fuel-mix at CERS Moldovenească for the year 2020 can be found at:
+# http://moldgres.com/wp-content/uploads/2021/02/proizvodstvennye-pokazateli-zao-moldavskaja-gres-za-2020-god.pdf
+
+# Annual reports from moldelectrica can be found at:
+# https://moldelectrica.md/ro/network/annual_report
+
+
+def template_price_response(zone_key, datetime, price) -> dict:
+    return {
+        'zoneKey': zone_key,
+        'datetime': datetime,
+        'currency': 'MDL',
+        'price': price,
+        'source': 'moldelectrica.md',
+    }
+
+
+def template_consumption_response(zone_key, datetime, consumption) -> dict:
+    return {
+        'zoneKey': zone_key,
+        'datetime': datetime,
+        'consumption': consumption,
+        'source': 'moldelectrica.md',
+    }
+
+
+def template_production_response(zone_key, datetime, production) -> dict:
+    return {
+        'zoneKey': zone_key,
+        'datetime': datetime,
+        'production': production,
+        'storage': {},
+        'source': 'moldelectrica.md',
+    }
+
+
+def template_exchange_response(sorted_zone_keys, datetime, netflow) -> dict:
+    return {
+        'sortedZoneKeys': sorted_zone_keys,
+        'datetime': datetime,
+        'netFlow': netflow,
+        'source': 'moldelectrica.md',
+    }
 
 
-def get_data(session=None):
-    """ Returns generation data as a list of floats."""
+def get_archive_data(session=None, dates=None) -> list:
+    """
+    Returns archive data as a list of ArchiveDatapoint.
 
+    Optionally accepts a date to fetch the data for,
+    or two dates specifing the range to fetch the data for.
+    Specifying a date-range too high will cause errors with the archive-server.
+    If no dates are specified data for the last 24 hours is fetched.
+    """
     s = session or requests.Session()
 
-    #In order for the data url to return data, cookies from the display url must be obtained then reused.
+    try:
+        date1, date2 = sorted(dates)
+    except:
+        date1 = date2 = dates
+
+    archive_url = archive_base_url
+    if date1 and date2:
+        archive_url += '&date1={}'.format(arrow.get(date1).to('Europe/Chisinau').format(archive_date_format))
+        archive_url += '&date2={}'.format(arrow.get(date2).to('Europe/Chisinau').format(archive_date_format))
+
+    data_response = s.get(archive_url, verify=False)
+    data = data_response.json()
+    try:
+        return [ArchiveDatapoint(arrow.get(entry[0], archive_datetime_format, tzinfo='Europe/Chisinau').datetime, *map(float, entry[1:])) for entry in data]
+    except:
+        raise Exception('Not able to parse received data. Check that the specifed URL returns correct data.')
+
+
+def get_data(session=None) -> list:
+    """Returns data as a list of floats."""
+    s = session or requests.Session()
+
+    # In order for data_url to return data, cookies from display_url must be obtained then reused.
     response = s.get(display_url, verify=False)
     data_response = s.get(data_url, verify=False)
     raw_data = data_response.text
     try:
-        data = [float(i) for i in raw_data.split(',')]
+        data = [float(i) if i else None for i in raw_data.split(',')]
     except:
-        raise Exception("Not able to parse received data. Check that the specifed URL returns correct data.")
+        raise Exception('Not able to parse received data. Check that the specifed URL returns correct data.')
 
     return data
 
 
+def fetch_price(zone_key='MD', session=None, target_datetime=None, logger=None) -> dict:
+    """
+    Returns the static price of electricity as specified here:
+    https://moldelectrica.md/ro/activity/tariff
+    It is defined by the following government-agency decision,
+    which is still in effect at the time of writing this (July 2021):
+    http://lex.justice.md/viewdoc.php?action=view&view=doc&id=360109&lang=1
+    """
+    if target_datetime:
+        raise NotImplementedError('This parser is not yet able to parse past dates for price')
+
+    dt = arrow.now('Europe/Chisinau').datetime
+    return template_price_response(zone_key, dt, '0.145')
+
+
+def fetch_consumption(zone_key='MD', session=None, target_datetime=None, logger=None) -> dict:
+    """Requests the consumption (in MW) of a given country."""
+    if target_datetime:
+        archive_data = get_archive_data(session, target_datetime)
+
+        datapoints = []
+        for entry in archive_data:
+            datapoint = template_consumption_response(zone_key, entry.datetime, entry.consumption)
+            datapoints.append(datapoint)
+        return datapoints
+    else:
+        field_values = get_data(session)
+
+        consumption = field_values[other_fields[0]['index']]
+
+        dt = arrow.now('Europe/Chisinau').datetime
+
+        datapoint = template_consumption_response(zone_key, dt, consumption)
+
+        return datapoint
+
+
 def fetch_production(zone_key='MD', session=None, target_datetime=None, logger=None) -> dict:
-    """Requests the last known production mix (in MW) of a given country."""
+    """Requests the production mix (in MW) of a given country."""
     if target_datetime:
-        raise NotImplementedError('This parser is not yet able to parse past dates')
+        archive_data = get_archive_data(session, target_datetime)
 
-    grid_status = get_data(session=session)
-    production = {'solar': None, 'wind': None, 'biomass': None, 'nuclear': 0.0}
+        datapoints = []
+        for entry in archive_data:
+            production = {'solar': None, 'wind': None, 'biomass': 0.0, 'nuclear': 0.0, 'gas': 0.0, 'hydro': 0.0}
 
-    production['gas'] = sum(itemgetter(3, 5, 6)(grid_status))
-    production['hydro'] = sum(itemgetter(2, 4)(grid_status))
-    production['unknown'] = grid_status[7]
+            production['gas'] += entry.tpp
+            production['hydro'] += entry.hpp
+            # Renewables (solar + biogas + wind) make up a small part of the energy produced.
+            # The exact mix of renewable enegry sources is unknown,
+            # so everything is attributed to biomass.
+            production['biomass'] += entry.res
 
-    consumption = grid_status[-5]
+            datapoint = template_production_response(zone_key, entry.datetime, production)
+            datapoints.append(datapoint)
+        return datapoints
+    else:
+        field_values = get_data(session)
 
-    dt = arrow.now('Europe/Chisinau').datetime
+        production = {'solar': None, 'wind': None, 'biomass': 0.0, 'nuclear': 0.0, 'gas': 0.0, 'hydro': 0.0}
 
-    datapoint = {
-        'zoneKey': zone_key,
-        'datetime': dt,
-        'consumption': consumption,
-        'production': production,
-        'storage': {},
-        'source': 'moldelectrica.md'
-    }
+        non_renewables_production = 0.0
+        for field in production_fields:
+            produced = field_values[field['index']]
+            non_renewables_production += produced
+            production[field['type']] += produced
+
+        # Renewables (solar + biogas + wind) make up a small part of the energy produced.
+        # They do not have an explicit entry,
+        # hence the difference between the actual generation and
+        # the sum of all other sectors are the renewables.
+        # The exact mix of renewable enegry sources is unknown,
+        # so everything is attributed to biomass.
+        production['biomass'] = field_values[other_fields[1]['index']] - non_renewables_production
+
+        dt = arrow.now('Europe/Chisinau').datetime
 
-    return datapoint
+        datapoint = template_production_response(zone_key, dt, production)
+
+        return datapoint
 
 
 def fetch_exchange(zone_key1, zone_key2, session=None, target_datetime=None, logger=None) -> dict:
     """Requests the last known power exchange (in MW) between two countries."""
-    if target_datetime:
-        raise NotImplementedError('This parser is not yet able to parse past dates')
+    sorted_zone_keys = '->'.join(sorted([zone_key1, zone_key2]))
 
-    sortedZoneKeys = '->'.join(sorted([zone_key1, zone_key2]))
+    if target_datetime:
+        archive_data = get_archive_data(session, target_datetime)
 
-    exchange_status = get_data(session=session)
+        datapoints = []
+        for entry in archive_data:
+            if sorted_zone_keys == 'MD->UA':
+                netflow = -1 * entry.exchange_UA_to_MD
+            elif sorted_zone_keys == 'MD->RO':
+                netflow = -1 * entry.exchange_RO_to_MD
+            else:
+                raise NotImplementedError('This exchange pair is not implemented')
 
-    if sortedZoneKeys == 'MD->UA':
-        netflow = -1 * exchange_status[-3]
-    elif sortedZoneKeys == 'MD->RO':
-        netflow = -1 * exchange_status[-2]
+            datapoint = template_exchange_response(sorted_zone_keys, entry.datetime, netflow)
+            datapoints.append(datapoint)
+        return datapoints
     else:
-        raise NotImplementedError('This exchange pair is not implemented')
+        field_values = get_data(session)
 
-    dt = arrow.now('Europe/Chisinau').datetime
+        if sorted_zone_keys == 'MD->UA':
+            netflow = -1 * field_values[other_fields[2]['index']]
+        elif sorted_zone_keys == 'MD->RO':
+            netflow = -1 * field_values[other_fields[3]['index']]
+        else:
+            raise NotImplementedError('This exchange pair is not implemented')
 
-    exchange = {
-        'sortedZoneKeys': sortedZoneKeys,
-        'datetime': dt,
-        'netFlow': netflow,
-        'source': 'moldelectrica.md'
-    }
+        dt = arrow.now('Europe/Chisinau').datetime
 
-    return exchange
+        datapoint = template_exchange_response(sorted_zone_keys, dt, netflow)
+
+        return datapoint
 
 
 if __name__ == '__main__':
     """Main method, never used by the Electricity Map backend, but handy for testing."""
 
-    print('fetch_production() ->')
-    print(fetch_production())
-    print('fetch_exchange(MD, UA) ->')
-    print(fetch_exchange('MD', 'UA'))
-    print('fetch_exchange(MD, RO) ->')
-    print(fetch_exchange('MD', 'RO'))
+    def try_print(callable, *args, **kwargs):
+        try:
+            result = callable(*args, **kwargs)
+            try:
+                print("[{}, ... ({} more elements)]".format(result[0], len(result)))
+            except:
+                print(result)
+        except Exception as e:
+            print(repr(e))
+
+    for target_datetime in (None, "2021-07-25T15:00"):
+        print('For target_datetime {}:'.format(target_datetime))
+        print('fetch_price() ->')
+        try_print(fetch_price, target_datetime=target_datetime)
+        print('fetch_consumption() ->')
+        try_print(fetch_consumption, target_datetime=target_datetime)
+        print('fetch_production() ->')
+        try_print(fetch_production, target_datetime=target_datetime)
+        print('fetch_exchange(MD, UA) ->')
+        try_print(fetch_exchange, 'MD', 'UA', target_datetime=target_datetime)
+        print('fetch_exchange(MD, RO) ->')
+        try_print(fetch_exchange, 'MD', 'RO', target_datetime=target_datetime)
+        print('------------')


Test patch: