Repository: electricitymaps/electricitymaps-contrib

Issue:
ENTSO-E electricity prices: change parser for DE and AT
In a nutshell: the ENTSO-E parser needs to be fixed to fetch the correct day-ahead hourly prices for Germany and Austria (instead of "no price" for Germany and "15 min EXAA auction prices" for Austria) using the data for the bidding zone (AT-DE-LU).

_____________

Here is the longer story:

Not sure if this hasn't already been discussed or you are aware, but couldn't find the right issue while searching:
The following image made me wonder about the displayed electricity prices for Austria:

![image](https://user-images.githubusercontent.com/25743609/36077589-faf54e10-0f6c-11e8-8d1b-e438f4c3bff2.png)

First off, the Austrian 15-min electricity price that is used right now should definitely not be used for electricitymap. Hourly day-ahead prices, like for the other countries, are way more representative as a graph, covering average hourly offer/demand situations and the most trading volume of the market. You will find the 15min values following the hourly prices, but still, there is a huge difference what hourly and quarter hourly prices "are made of". And EEX/EPEX Spot and EXAA are two different market places with different trading volumes, too.

In general, I think, this comes from the parser currently using the "country" (AT) data from ENTSO-E.
This is also why there is no price for DE - because the hourly (60 min block) day-ahead price is only given for the entire bidding zone AT-DE-LU, not seperately for German area (n/e).
As you may know, AT and DE are in the same bidding zone for day-ahead auctions (with LUxemburg), so **day-ahead hourly prices are always equal in these three countries**, without any exception.

At the moment, the electricity prices for AT seem to come from the 15-min day-ahead auction from the Austrian power exchange EXAA. The zig-zagging prices and 15 min-intervals in the image definitely looked very much like that. Looking it up on the [EXAA site](https://www.exaa.at/de/marktdaten/handelsergebnisse) confirmed this theory 100%.

The **quarter-hourly day-ahead blocks** are **only** used to compensate the biggest part of the resulting inter-hourly surplusses or deficits from the hourly day-ahead auctions. This is also why they have this predictable zig-zag shape. 
(((Just to make the list complete: On the spot market (short term trading, no auction!), quarter-hourly _intraday_ trading is used to compensate intraday deviations of demand or generation (e.g. less/more wind, solar, broken unit, ...)))


The hourly data is on ENTSO-E for the entire bidding zone Austria/Germany/(Luxembourg):
![image](https://user-images.githubusercontent.com/25743609/36078029-dddd3acc-0f71-11e8-9c3d-5472dbb508d5.png)

Just not for Germany in the "country" section:
![image](https://user-images.githubusercontent.com/25743609/36078038-f4452022-0f71-11e8-9e07-0cdb41c068fd.png)

And for Austria you have quarter hour prices (EXAA 15 min auction):
![image](https://user-images.githubusercontent.com/25743609/36078045-0b07f5dc-0f72-11e8-9c12-82f7ad85ad80.png)


Discussion:
Good point, very clear!
We should indeed remove the existing AT 15min-ahead prices from the map and only use day-ahead spot to stay consistent, and display this price for both DE and AT and the map
Hi guys, anyone working on this? If not, I would be happy to.
@lorrieq nobody is working on this now to my knowledge so feel free to have a go. Let us know if you have any questions.
I've taken an initial look. So Austria, Germany and Luxembourg prices need to use the bidding zone 10Y1001A1001A63L. I think it will be necessary to add some override in fetch_price like is done in fetch_exchange with ENTSOE_EXCHANGE_DOMAIN_OVERRIDE:

![image](https://user-images.githubusercontent.com/6525086/37296751-ce143560-2613-11e8-8e28-828426e3d3c0.png)

My suggestion is to add a constant ENTSOE_PRICE_DOMAIN_OVERRIDE and then add code to fetch_price to select the domain from ENTSOE_PRICE_DOMAIN_OVERRIDE if the zone_key exists there and if not to select the domain from the usual ENTSOE_DOMAIN_MAPPINGS.

Anyone see any problems with this approach?
Sounds good.
Just to be sure: doesn't Germany have two zones and thus using one price
for both would result in an approximation?

On Mar 12, 2018 17:44, "Laurence Quinn" <notifications@github.com> wrote:

> I've taken an initial look. So Austria, Germany and Luxembourg prices need
> to use the bidding zone 10Y1001A1001A63L. I think it will be necessary to
> add some override in fetch_price like is done in fetch_exchange with
> ENTSOE_EXCHANGE_DOMAIN_OVERRIDE:
>
> [image: image]
> <https://user-images.githubusercontent.com/6525086/37296751-ce143560-2613-11e8-8e28-828426e3d3c0.png>
>
> My suggestion is to add a constant ENTSOE_PRICE_DOMAIN_OVERRIDE and then
> add code to fetch_price to select the domain from
> ENTSOE_PRICE_DOMAIN_OVERRIDE if the zone_key exists there and if not to
> select the domain from the usual ENTSOE_DOMAIN_MAPPINGS.
>
> Anyone see any problems with this approach?
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/tmrowco/electricitymap/issues/1086#issuecomment-372378344>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABlEKHobG-6rvITdd4LY_KNPDLPraZrtks5tdqXUgaJpZM4SBgHU>
> .
>

@corradio Germany is one single price zone, so no worries. There were only debates about splitting it in the past due to transmission bottlenecks and other issues
OK thanks for the clarification!

On Mon 12 Mar 2018 at 19:39, Alex <notifications@github.com> wrote:

> @corradio <https://github.com/corradio> Germany is one single price zone,
> so no worries. There were only debates about splitting it in the past due
> to transmission bottlenecks and other issues
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/tmrowco/electricitymap/issues/1086#issuecomment-372418963>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABlEKLkSGo42yLYkhRfGdXp39QrllIY6ks5tdsDMgaJpZM4SBgHU>
> .
>

Hi guys,
I have that implemented but I am having some issues trying to push my remote branch to GitHub for a pull request. Initially I cloned electricitymap with HTTPS so my origin remote was _https://github.com/tmrowco/electricitymap.git_ and the error I got when pushing was:

<img width="753" alt="screen shot 2018-03-13 at 22 48 40" src="https://user-images.githubusercontent.com/6525086/37373944-d5998148-2710-11e8-831c-d62ae931cf81.png">

Then after some reading online I set up SSH on my Mac and GitHub but got the same error. Some more reading made me change the origin remote to SSH, _git@github.com:tmrowco/electricitymap.git_, but I got a different error when pushing:

<img width="473" alt="screen shot 2018-03-13 at 22 47 02" src="https://user-images.githubusercontent.com/6525086/37373880-8173fac6-2710-11e8-8718-d1434255176b.png">

Any advice would be great. Sorry for creating an issue within an issue!
Hi @lorrieq - you need to fork the repository, push that new branch there, and then create a pull request! Ping us on slack if you have any troubles

Resolving patch:
diff --git a/parsers/ENTSOE.py b/parsers/ENTSOE.py
--- a/parsers/ENTSOE.py
+++ b/parsers/ENTSOE.py
@@ -107,6 +107,13 @@
     'PL->UA': [ENTSOE_DOMAIN_MAPPINGS['PL'], '10Y1001A1001A869']
 }
 
+# Some zone_keys are part of bidding zone domains for price data
+ENTSOE_PRICE_DOMAIN_OVERRIDE = {
+    'AT': '10Y1001A1001A63L',
+    'DE': '10Y1001A1001A63L',
+    'LU': '10Y1001A1001A63L'
+}
+
 
 class QueryError(Exception):
     """Raised when a query to ENTSOE returns no matching data."""
@@ -707,7 +714,10 @@ def fetch_price(zone_key, session=None, target_datetime=None, logger=None):
     # Note: This is day-ahead prices
     if not session:
         session = requests.session()
-    domain = ENTSOE_DOMAIN_MAPPINGS[zone_key]
+    if zone_key in ENTSOE_PRICE_DOMAIN_OVERRIDE:
+        domain = ENTSOE_PRICE_DOMAIN_OVERRIDE[zone_key]
+    else:
+        domain = ENTSOE_DOMAIN_MAPPINGS[zone_key]
     # Grab consumption
     parsed = parse_price(query_price(domain, session, target_datetime=target_datetime))
     if parsed:


Test patch: