Repository: electricitymaps/electricitymaps-contrib

Issue:
Price graph is 1hr ahead
The French power price graph is one hour ahead compared to the surrounding countries. I suppose this is an easy time-zone related problem.

Look at this graph:
<img width="347" alt="screenshot 2018-03-27 16 11 40" src="https://user-images.githubusercontent.com/5185181/37972963-becff93c-31d9-11e8-9d73-bb89e5923fd6.png">

And compare it with Germany (currently they match):
<img width="341" alt="screenshot 2018-03-27 16 11 53" src="https://user-images.githubusercontent.com/5185181/37973056-f26d24ae-31d9-11e8-8143-c2f835c84b20.png">

Spain:
<img width="337" alt="screenshot 2018-03-27 16 15 22" src="https://user-images.githubusercontent.com/5185181/37973124-1ac548b4-31da-11e8-8c48-38e4810a289f.png">

Belgium:
<img width="343" alt="screenshot 2018-03-27 16 15 32" src="https://user-images.githubusercontent.com/5185181/37973136-21bacb1c-31da-11e8-8e78-bfdde863fb77.png">



Discussion:
I can confirm this: France shows the electricity price for the ongoing hour (16-17 CET - this should be the favourable option), whereas for the other countries, the price of the previous hour (15-16 CET) is displayed. Maybe it's related to summer time? Not sure how it was previously.

Some direct links to the data:
[LINK to bidding zone FR](https://transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=TABLE&areaType=BZN&atch=false&dateTime.dateTime=27.03.2018+00:00|CET|DAY&biddingZone.values=CTY|10YFR-RTE------C!BZN|10YFR-RTE------C&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2))

[LINK to bidding zone DE-AT-LU](https://transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=TABLE&areaType=BZN&atch=false&dateTime.dateTime=27.03.2018+00:00|CET|DAY&biddingZone.values=CTY|10Y1001A1001A83F!BZN|10Y1001A1001A63L&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2))

[LINK to bidding zone ES](https://transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=TABLE&areaType=BZN&atch=false&dateTime.dateTime=27.03.2018+00:00|CET|DAY&biddingZone.values=CTY|10YES-REE------0!BZN|10YES-REE------0&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2))
Or is this related to the granularity of the data and FR using RTE as source, which discloses data faster/earlier?
Probably there's a bug in the RTE parser.
My suggestion: remove the price section of the RTE parser, and use ENTSOE
instead.

Olivier

On Tue, Mar 27, 2018 at 4:39 PM, Alex <notifications@github.com> wrote:

> Or is this related to the granularity of the data and FR using RTE as
> source, which discloses data faster/earlier?
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/tmrowco/electricitymap/issues/1272#issuecomment-376549888>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABlEKOFSDwhqgGTwFzR21jiQVTvNtwEzks5tik8WgaJpZM4S9ApA>
> .
>

@corradio the ENTSO-E data that is used for the live-view is a bit behind the RTE data, so the displayed prices from ENTSO-E are kind of "1 hour behind"
I all ready noticed the problem more than one week ago, but didn't yet report it. So it's not related to summer time. 

I took these screenshot (not entirely sure which countries) 20th of march:
<img width="344" alt="screenshot 2018-03-20 17 08 57" src="https://user-images.githubusercontent.com/5185181/37976491-e8531d90-31e1-11e8-9b6b-1644d9f01b83.png">
<img width="338" alt="screenshot 2018-03-20 17 09 20" src="https://user-images.githubusercontent.com/5185181/37976499-ec0c8aac-31e1-11e8-9945-8325b66bc67d.png">

@alixunderplatz can you detail a bit what you mean by saying ENTSO-E data is behind RTE data?
You mean that the timestamps shown here
https://transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=TABLE&areaType=BZN&atch=false&dateTime.dateTime=28.03.2018+00:00|CET|DAY&biddingZone.values=CTY|10YFR-RTE------C!BZN|10YFR-RTE------C&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2)
are not the correct ones?
@corradio Sure: Let's say we choose the price data for this recent hour 12:00-12:59 h CET.
For France, the correct day-ahead market price of 49.62 €/MWh is shown in the graph already.

Generation data for this period of time is also there.
![image](https://user-images.githubusercontent.com/25743609/38024623-e28ce19e-3285-11e8-984d-bab2d6d67572.png)
_ _ _ _

For bidding zone Germany/Austria/Luxembourg, still the price of the hour 11:00-11:59 is shown (44.68 instead of 41.84), although my cursor is behind the 12h mark - so there is something wrong here. Maybe the timestamp or timezone used in the electricitymap is kind of wrong and shifted one hour into the future.

**Or**, one could assume this price value is assigned, because there is no complete set of generation data yet for the time between 12:00 to 12:59. The last available complete values from 11:30 are "extended" into the 12th hour, and so is the price. But it's hard to see in the tiny price box and also wouldn't make sense

![image](https://user-images.githubusercontent.com/25743609/38024674-fb6b397c-3285-11e8-90ee-3d2644a52b67.png)
I can confirm there is nothing wrong with the data on ENTSO-E. Due to the market coupling mechanisms, some hours may have the same market prices for both bidding zones FR and DE/AT/LUX. For these two bidding zones, you can see prices for today are equal between 2 and 5 at night, which proves everything is okay there.
I've looked at the data behind the history graph, and I saw the following for DK-DK2:
- `stateDatetime`: 2018-03-27T11:00:00.000Z
- value: 47.53

On ENTSOE, using UTC: 
- 10:00 - 11:00 -> 47.53
- 11:00 - 12:00 -> 45.13

So it seems like we're off by one hour indeed.


Resolving patch:
diff --git a/parsers/ENTSOE.py b/parsers/ENTSOE.py
--- a/parsers/ENTSOE.py
+++ b/parsers/ENTSOE.py
@@ -419,12 +419,12 @@ def query_consumption_forecast(in_domain, session, target_datetime=None):
 def datetime_from_position(start, position, resolution):
     """Finds time granularity of data."""
 
-    m = re.search('PT(\d+)([M])', resolution)
+    m = re.search(r'PT(\d+)([M])', resolution)
     if m:
         digits = int(m.group(1))
         scale = m.group(2)
         if scale == 'M':
-            return start.replace(minutes=position * digits)
+            return start.replace(minutes=(position - 1) * digits)
     raise NotImplementedError('Could not recognise resolution %s' % resolution)
 
 


Test patch: