Repository: rotki/rotki
Pull request #9566: improve per-asset historical prices endpoint
Author: login: prettyirrelevant
Labels: name: ready for final review
description: Backend PR ready to be reviewed by great Lefteris
Created: 2025-03-05T20:52:20Z | Merged: 2025-03-06T13:49:44Z

- add `exclude_timestamps` to request.
- filter out future timestamps when generating timestamps
- separate between no prices found and ratelimit errors.


Diff:
diff --git a/docs/api.rst b/docs/api.rst
index 00c7cb9745e..840664cccd1 100644
--- a/docs/api.rst
+++ b/docs/api.rst
@@ -14149,7 +14149,8 @@ Historical Balance Queries
     :reqjsonarr integer interval: The time interval between price queries in seconds
     :reqjsonarr integer from_timestamp: The start timestamp of the query range
     :reqjsonarr integer to_timestamp: The end timestamp of the query range
-     :reqjson integer only_cache_period: (Optional) Time period in seconds around each timestamp to search for cached prices. If omitted, falls back to querying historical prices
+    :reqjsonarr integer exclude_timestamps: (Optional) List of timestamps to exclude from price queries.
+    :reqjson integer only_cache_period: (Optional) Time period in seconds around each timestamp to search for cached prices. If omitted, falls back to querying historical prices
     :reqjsonarr boolean async_query: (Optional) Whether to process the request asynchronously
 
     **Example Response:**
@@ -14167,13 +14168,15 @@ Historical Balance Queries
               "1672617600": "19500",
               "1672704000": "19800"
             },
-            "no_prices_timestamps": [1672790400, 1672876800]
+            "no_prices_timestamps": [1672790400, 1672876800],
+            "rate_limited_prices_timestamps": [1672790400, 1672876800]
           },
           "status_code": 200
         }
 
       :resjson object prices: Mapping of timestamps to price values in user's profit currency
       :resjson list[integer] no_prices_timestamps: List of timestamps where price data was not available
+      :resjson list[integer] rate_limited_prices_timestamps: List of timestamps where price queries were rate-limited by the data provider
       :statuscode 200: Historical prices returned
       :statuscode 400: Malformed query
       :statuscode 401: User is not logged in
diff --git a/rotkehlchen/api/rest.py b/rotkehlchen/api/rest.py
index ecfc87a1154..6ca42b5212a 100644
--- a/rotkehlchen/api/rest.py
+++ b/rotkehlchen/api/rest.py
@@ -5278,16 +5278,21 @@ def get_historical_prices_per_asset(
             interval: int,
             to_timestamp: Timestamp,
             from_timestamp: Timestamp,
+            exclude_timestamps: set[Timestamp],
             only_cache_period: int | None = None,
     ) -> dict[str, Any]:
         prices = {}
         no_prices_ts: list[Timestamp] = []
+        rate_limited_prices_ts: list[Timestamp] = []
         with (db := self.rotkehlchen.data.db).conn.read_ctx() as cursor:
             main_currency = db.get_setting(cursor=cursor, name='main_currency')
 
-        total_intervals = (to_timestamp - from_timestamp) // interval + 1
-        timestamps = [Timestamp(from_timestamp + (i * interval)) for i in range(total_intervals)]
-
+        # Generate timestamps at regular intervals up to current time or
+        # to_timestamp (whichever comes first). Skip any timestamps found in exclude_timestamps
+        timestamps = [
+            ts for i in range((min(ts_now(), to_timestamp) - from_timestamp) // interval + 1)
+            if (ts := Timestamp(from_timestamp + (i * interval))) not in exclude_timestamps
+        ]
         if only_cache_period is not None:
             for price_result in GlobalDBHandler.get_historical_prices(
                 query_data=[(asset, main_currency, ts) for ts in timestamps],
@@ -5299,8 +5304,10 @@ def get_historical_prices_per_asset(
             return _wrap_in_ok_result(result={
                 'prices': prices,
                 'no_prices_timestamps': no_prices_ts,
+                'rate_limited_prices_timestamps': rate_limited_prices_ts,
             })
 
+        total = len(timestamps)
         for processed_count, current_ts in enumerate(timestamps, 1):
             try:
                 price = PriceHistorian.query_historical_price(
@@ -5308,8 +5315,11 @@ def get_historical_prices_per_asset(
                     to_asset=main_currency,
                     timestamp=current_ts,
                 )
-            except NoPriceForGivenTimestamp:
-                no_prices_ts.append(current_ts)
+            except NoPriceForGivenTimestamp as e:
+                if e.rate_limited:
+                    rate_limited_prices_ts.append(current_ts)
+                else:
+                    no_prices_ts.append(current_ts)
             else:
                 prices[current_ts] = str(price)
 
@@ -5317,7 +5327,7 @@ def get_historical_prices_per_asset(
                 db.msg_aggregator.add_message(
                     message_type=WSMessageType.PROGRESS_UPDATES,
                     data={
-                        'total': total_intervals,
+                        'total': total,
                         'processed': processed_count,
                         'subtype': str(ProgressUpdateSubType.HISTORICAL_PRICE_QUERY_STATUS),
                     },
@@ -5326,4 +5336,5 @@ def get_historical_prices_per_asset(
         return _wrap_in_ok_result(result={
             'prices': prices,
             'no_prices_timestamps': no_prices_ts,
+            'rate_limited_prices_timestamps': rate_limited_prices_ts,
         })
diff --git a/rotkehlchen/api/v1/resources.py b/rotkehlchen/api/v1/resources.py
index b410b64dff9..1e4059baec5 100644
--- a/rotkehlchen/api/v1/resources.py
+++ b/rotkehlchen/api/v1/resources.py
@@ -3345,6 +3345,7 @@ def post(
             async_query: bool,
             from_timestamp: Timestamp,
             to_timestamp: Timestamp,
+            exclude_timestamps: set[Timestamp],
             only_cache_period: int | None = None,
     ) -> Response:
         return self.rest_api.get_historical_prices_per_asset(
@@ -3354,4 +3355,5 @@ def post(
             to_timestamp=to_timestamp,
             from_timestamp=from_timestamp,
             only_cache_period=only_cache_period,
+            exclude_timestamps=exclude_timestamps,
         )
diff --git a/rotkehlchen/api/v1/schemas.py b/rotkehlchen/api/v1/schemas.py
index 0ff5f1df259..4d9a1ba0d4f 100644
--- a/rotkehlchen/api/v1/schemas.py
+++ b/rotkehlchen/api/v1/schemas.py
@@ -3870,6 +3870,7 @@ class HistoricalPricesPerAssetSchema(AsyncQueryArgumentSchema, TimestampRangeSch
         load_default=None,
         validate=webargs.validate.Range(min=1, error='Cache period must be a positive integer'),
     )
+    exclude_timestamps = fields.List(TimestampField(), load_default=list)
     asset = AssetField(expected_type=Asset, required=True)
 
     @validates_schema
@@ -3896,6 +3897,7 @@ def transform_data(
         - Rounds up `to_timestamp` to nearest multiple of interval
         """
         interval = data['interval']
+        data['exclude_timestamps'] = set(data['exclude_timestamps'])
         data['from_timestamp'] = (data['from_timestamp'] // interval) * interval
         data['to_timestamp'] = ((data['to_timestamp'] + interval - 1) // interval) * interval
 
diff --git a/rotkehlchen/tests/api/test_historical_balances.py b/rotkehlchen/tests/api/test_historical_balances.py
index 2c3877f4a70..604f15b7d50 100644
--- a/rotkehlchen/tests/api/test_historical_balances.py
+++ b/rotkehlchen/tests/api/test_historical_balances.py
@@ -1,5 +1,6 @@
 from http import HTTPStatus
 from typing import TYPE_CHECKING
+from unittest.mock import patch
 
 import pytest
 import requests
@@ -10,6 +11,7 @@
 from rotkehlchen.constants import DAY_IN_SECONDS, ONE
 from rotkehlchen.constants.assets import A_BTC, A_ETH, A_EUR
 from rotkehlchen.db.history_events import DBHistoryEvents
+from rotkehlchen.errors.price import NoPriceForGivenTimestamp
 from rotkehlchen.exchanges.data_structures import Trade
 from rotkehlchen.fval import FVal
 from rotkehlchen.history.events.structures.asset_movement import AssetMovement
@@ -18,6 +20,7 @@
     HistoryEventSubType,
     HistoryEventType,
 )
+from rotkehlchen.history.price import PriceHistorian
 from rotkehlchen.history.types import HistoricalPrice, HistoricalPriceOracle
 from rotkehlchen.tests.utils.api import (
     api_url_for,
@@ -28,13 +31,14 @@
 )
 from rotkehlchen.tests.utils.constants import A_DASH
 from rotkehlchen.types import AssetAmount, ChainID, Location, Price, Timestamp, TradeType
-from rotkehlchen.utils.misc import timestamp_to_daystart_timestamp, ts_sec_to_ms
+from rotkehlchen.utils.misc import timestamp_to_daystart_timestamp, ts_now, ts_sec_to_ms
 
 if TYPE_CHECKING:
     from rotkehlchen.api.server import APIServer
     from rotkehlchen.globaldb.handler import GlobalDBHandler
 
 START_TS = Timestamp(1672531200)
+DAY_AFTER_START_TS = Timestamp(START_TS + DAY_IN_SECONDS)
 
 
 @pytest.fixture(name='setup_historical_data')
@@ -128,7 +132,7 @@ def test_get_historical_balance(
             'timestamphistoricalbalanceresource',
         ),
         json={
-            'timestamp': START_TS + DAY_IN_SECONDS,  # Day 2
+            'timestamp': DAY_AFTER_START_TS,  # Day 2
             'async_query': True,
         },
     )
@@ -158,7 +162,7 @@ def test_get_historical_asset_balance(
             write_cursor=write_cursor,
             trades=[
                 Trade(
-                    timestamp=Timestamp(START_TS + DAY_IN_SECONDS),  # Day 2
+                    timestamp=DAY_AFTER_START_TS,  # Day 2
                     location=Location.EXTERNAL,
                     base_asset=A_BTC,
                     quote_asset=A_EUR,
@@ -462,7 +466,7 @@ def test_get_historical_netvalue(
             write_cursor=write_cursor,
             trades=[
                 Trade(
-                    timestamp=Timestamp(START_TS + DAY_IN_SECONDS),  # Day 2
+                    timestamp=DAY_AFTER_START_TS,  # Day 2
                     location=Location.EXTERNAL,
                     base_asset=A_BTC,
                     quote_asset=A_EUR,
@@ -502,7 +506,7 @@ def test_get_historical_netvalue(
 
     expected_timestamps = [
         timestamp_to_daystart_timestamp(START_TS),  # Day 1
-        timestamp_to_daystart_timestamp(Timestamp(START_TS + DAY_IN_SECONDS)),  # Day 2
+        timestamp_to_daystart_timestamp(DAY_AFTER_START_TS),  # Day 2
         timestamp_to_daystart_timestamp(Timestamp(START_TS + DAY_IN_SECONDS * 2)),  # Day 3
     ]
     expected_values = [
@@ -723,7 +727,7 @@ def test_get_historical_prices_per_asset(
         json={
             'asset': 'INVALID',
             'from_timestamp': START_TS,
-            'to_timestamp': START_TS + DAY_IN_SECONDS,
+            'to_timestamp': DAY_AFTER_START_TS,
             'interval': DAY_IN_SECONDS,
         },
     )
@@ -767,13 +771,13 @@ def test_get_historical_prices_per_asset(
         json={
             'asset': 'eip155:8453/erc20:0x5875eEE11Cf8398102FdAd704C9E96607675467a',
             'from_timestamp': START_TS,
-            'to_timestamp': START_TS + DAY_IN_SECONDS,
+            'to_timestamp': DAY_AFTER_START_TS,
             'interval': DAY_IN_SECONDS,
         },
     )
     result = assert_proper_sync_response_with_result(response)
     assert result['prices'] == {}
-    assert result['no_prices_timestamps'] == [START_TS, START_TS + DAY_IN_SECONDS]
+    assert result['no_prices_timestamps'] == [START_TS, DAY_AFTER_START_TS]
 
     # invalid from_timestamp > to_timestamp
     response = requests.post(
@@ -783,7 +787,7 @@ def test_get_historical_prices_per_asset(
         ),
         json={
             'asset': 'BTC',
-            'from_timestamp': START_TS + DAY_IN_SECONDS,
+            'from_timestamp': DAY_AFTER_START_TS,
             'to_timestamp': START_TS,
             'interval': DAY_IN_SECONDS,
         },
@@ -805,7 +809,7 @@ def test_get_historical_prices_per_asset(
         from_asset=A_DASH,
         to_asset=A_EUR,
         source=HistoricalPriceOracle.CRYPTOCOMPARE,
-        timestamp=Timestamp(START_TS + DAY_IN_SECONDS),
+        timestamp=DAY_AFTER_START_TS,
         price=Price(FVal('20')),
     )])
     response = requests.post(
@@ -824,5 +828,98 @@ def test_get_historical_prices_per_asset(
     result = assert_proper_sync_response_with_result(response)
     assert result['prices'] == {
         str(START_TS): '10',
-        str(START_TS + DAY_IN_SECONDS): '20',
+        str(DAY_AFTER_START_TS): '20',
     }
+
+    # test future timestamp exclusion
+    current_time = ts_now()
+    response = requests.post(
+        api_url_for(
+            rotkehlchen_api_server,
+            'historicalpricesperassetresource',
+        ),
+        json={
+            'asset': 'BTC',
+            'from_timestamp': current_time - DAY_IN_SECONDS,
+            'to_timestamp': (future_ts := current_time + DAY_IN_SECONDS),  # Include timestamp in the future  # noqa: E501
+            'interval': DAY_IN_SECONDS,
+        },
+    )
+    result = assert_proper_sync_response_with_result(response)
+    assert str(future_ts) not in result['prices']
+    assert future_ts not in result['no_prices_timestamps']
+    assert future_ts not in result['rate_limited_prices_timestamps']
+    assert max(int(ts) for ts in result['prices']) <= current_time
+
+    # test excluded timestamps
+    response = requests.post(
+        api_url_for(
+            rotkehlchen_api_server,
+            'historicalpricesperassetresource',
+        ),
+        json={
+            'asset': 'BTC',
+            'from_timestamp': START_TS,
+            'to_timestamp': START_TS + 2 * DAY_IN_SECONDS,
+            'interval': DAY_IN_SECONDS,
+            'exclude_timestamps': [DAY_AFTER_START_TS],  # Exclude middle timestamp
+        },
+    )
+    result = assert_proper_sync_response_with_result(response)
+    assert str(START_TS) in result['prices']
+    assert str(DAY_AFTER_START_TS) not in result['prices']
+    assert str(START_TS + 2 * DAY_IN_SECONDS) in result['prices']
+    assert DAY_AFTER_START_TS not in result['no_prices_timestamps']
+
+    # Test both rate limited and no price found scenarios
+    original_query_historical_price = PriceHistorian.query_historical_price
+
+    def mock_query_historical_price(from_asset, to_asset, timestamp):
+        # Rate limit for one timestamp
+        if timestamp == DAY_AFTER_START_TS:
+            raise NoPriceForGivenTimestamp(
+                from_asset=from_asset,
+                to_asset=to_asset,
+                time=timestamp,
+                rate_limited=True,
+            )
+        # No price found for another timestamp
+        elif timestamp == START_TS + 2 * DAY_IN_SECONDS:
+            raise NoPriceForGivenTimestamp(
+                from_asset=from_asset,
+                to_asset=to_asset,
+                time=timestamp,
+                rate_limited=False,
+            )
+        # Normal behavior for other timestamps
+        return original_query_historical_price(from_asset, to_asset, timestamp)
+
+    with patch('rotkehlchen.history.price.PriceHistorian.query_historical_price', side_effect=mock_query_historical_price):  # noqa: E501
+        response = requests.post(
+            api_url_for(
+                rotkehlchen_api_server,
+                'historicalpricesperassetresource',
+            ),
+            json={
+                'asset': 'BTC',
+                'from_timestamp': START_TS,
+                'to_timestamp': START_TS + 3 * DAY_IN_SECONDS,
+                'interval': DAY_IN_SECONDS,
+            },
+        )
+        result = assert_proper_sync_response_with_result(response)
+        # The first timestamp should have a price
+        assert str(START_TS) in result['prices']
+
+        # The second timestamp should be rate limited
+        assert str(DAY_AFTER_START_TS) not in result['prices']
+        assert DAY_AFTER_START_TS not in result['no_prices_timestamps']
+        assert DAY_AFTER_START_TS in result['rate_limited_prices_timestamps']
+
+        # The third timestamp should have no price found (not rate limited)
+        assert str(START_TS + 2 * DAY_IN_SECONDS) not in result['prices']
+        assert START_TS + 2 * DAY_IN_SECONDS in result['no_prices_timestamps']
+        assert START_TS + 2 * DAY_IN_SECONDS not in result['rate_limited_prices_timestamps']
+
+        # The fourth timestamp should have a price (uses original function)
+        assert str(START_TS + 3 * DAY_IN_SECONDS) in result['prices']


Reviews:
user: login: yabirgb
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T08:58:07Z
user: login: prettyirrelevant
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T12:24:03Z
user: login: LefterisJP
body: LGTM
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-06T13:14:33Z
user: login: LefterisJP
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T13:17:48Z

Review comments:
diff_hunk: @@ -5278,15 +5278,21 @@ def get_historical_prices_per_asset(
             interval: int,
             to_timestamp: Timestamp,
             from_timestamp: Timestamp,
+            exclude_timestamps: set[Timestamp],
             only_cache_period: int | None = None,
     ) -> dict[str, Any]:
         prices = {}
         no_prices_ts: list[Timestamp] = []
+        rate_limited_prices_ts: list[Timestamp] = []
         with (db := self.rotkehlchen.data.db).conn.read_ctx() as cursor:
             main_currency = db.get_setting(cursor=cursor, name='main_currency')
 
-        total_intervals = (to_timestamp - from_timestamp) // interval + 1
-        timestamps = [Timestamp(from_timestamp + (i * interval)) for i in range(total_intervals)]
+        # Generate timestamps for each interval between from_timestamp and to_timestamp
+        # Filter out any timestamps that are in the exclude_timestamps or in the future
+        timestamps = [
+            ts for i in range((to_timestamp - from_timestamp) // interval + 1)
+            if (ts := Timestamp(from_timestamp + (i * interval))) not in exclude_timestamps and ts <= ts_now()  # noqa: E501
+        ]
path: rotkehlchen/api/rest.py
user: login: yabirgb
body: mmmm here I'm not sure if it is calling ts_now() for each time that you check in the loop. Move it outside the loop. 

Also instead of checkign ts <= ts_now you can do 
```suggestion
        timestamps = [
            ts for i in range((min(ts_now(), to_timestamp) - from_timestamp) // interval + 1)
            if (ts := Timestamp(from_timestamp + (i * interval))) not in exclude_timestamps and ts <= ts_now()  # noqa: E501
        ]
```
created_at: 2025-03-06T08:49:17Z
author_association: MEMBER
diff_hunk: @@ -5308,16 +5315,19 @@ def get_historical_prices_per_asset(
                     to_asset=main_currency,
                     timestamp=current_ts,
                 )
-            except NoPriceForGivenTimestamp:
-                no_prices_ts.append(current_ts)
+            except NoPriceForGivenTimestamp as e:
+                if e.rate_limited:
+                    rate_limited_prices_ts.append(current_ts)
+                else:
+                    no_prices_ts.append(current_ts)
             else:
                 prices[current_ts] = str(price)
 
             if processed_count % 10 == 0:  # Send progress update every 10 price queries
                 db.msg_aggregator.add_message(
                     message_type=WSMessageType.PROGRESS_UPDATES,
                     data={
-                        'total': total_intervals,
+                        'total': len(timestamps),
path: rotkehlchen/api/rest.py
user: login: yabirgb
body: calculate len(timestamps) outside the loop in a variable and use it here
created_at: 2025-03-06T08:51:47Z
author_association: MEMBER
diff_hunk: @@ -826,3 +829,96 @@ def test_get_historical_prices_per_asset(
         str(START_TS): '10',
         str(START_TS + DAY_IN_SECONDS): '20',
     }
+
+    # test future timestamp exclusion
+    current_time = ts_now()
+    future_ts = current_time + DAY_IN_SECONDS
+    response = requests.post(
+        api_url_for(
+            rotkehlchen_api_server,
+            'historicalpricesperassetresource',
+        ),
+        json={
+            'asset': 'BTC',
+            'from_timestamp': current_time - DAY_IN_SECONDS,
+            'to_timestamp': future_ts,  # Include timestamp in the future
+            'interval': DAY_IN_SECONDS,
+        },
+    )
+    result = assert_proper_sync_response_with_result(response)
+    assert str(future_ts) not in result['prices']
path: rotkehlchen/tests/api/test_historical_balances.py
user: login: yabirgb
body: maybe check that the max timestamp here is also current_time (or curernt time plus a bit)
created_at: 2025-03-06T08:52:34Z
author_association: MEMBER
diff_hunk: @@ -826,3 +829,96 @@ def test_get_historical_prices_per_asset(
         str(START_TS): '10',
         str(START_TS + DAY_IN_SECONDS): '20',
     }
+
+    # test future timestamp exclusion
+    current_time = ts_now()
+    future_ts = current_time + DAY_IN_SECONDS
+    response = requests.post(
+        api_url_for(
+            rotkehlchen_api_server,
+            'historicalpricesperassetresource',
+        ),
+        json={
+            'asset': 'BTC',
+            'from_timestamp': current_time - DAY_IN_SECONDS,
+            'to_timestamp': future_ts,  # Include timestamp in the future
+            'interval': DAY_IN_SECONDS,
+        },
+    )
+    result = assert_proper_sync_response_with_result(response)
+    assert str(future_ts) not in result['prices']
+    assert future_ts not in result['no_prices_timestamps']
+    assert future_ts not in result['rate_limited_prices_timestamps']
+
+    # test excluded timestamps
+    response = requests.post(
+        api_url_for(
+            rotkehlchen_api_server,
+            'historicalpricesperassetresource',
+        ),
+        json={
+            'asset': 'BTC',
+            'from_timestamp': START_TS,
+            'to_timestamp': START_TS + 2 * DAY_IN_SECONDS,
+            'interval': DAY_IN_SECONDS,
+            'exclude_timestamps': [START_TS + DAY_IN_SECONDS],  # Exclude middle timestamp
+        },
+    )
+    result = assert_proper_sync_response_with_result(response)
+    assert str(START_TS) in result['prices']
+    assert str(START_TS + DAY_IN_SECONDS) not in result['prices']
+    assert str(START_TS + 2 * DAY_IN_SECONDS) in result['prices']
+    assert START_TS + DAY_IN_SECONDS not in result['no_prices_timestamps']
+
+    # Test both rate limited and no price found scenarios
+    original_query_historical_price = PriceHistorian.query_historical_price
+
+    def mock_query_historical_price(from_asset, to_asset, timestamp):
+        # Rate limit for one timestamp
+        if timestamp == START_TS + DAY_IN_SECONDS:
+            raise NoPriceForGivenTimestamp(
+                from_asset=from_asset,
+                to_asset=to_asset,
+                time=timestamp,
+                rate_limited=True,
+            )
+        # No price found for another timestamp
+        elif timestamp == START_TS + 2 * DAY_IN_SECONDS:
+            raise NoPriceForGivenTimestamp(
+                from_asset=from_asset,
+                to_asset=to_asset,
+                time=timestamp,
+                rate_limited=False,
+            )
+        # Normal behavior for other timestamps
+        return original_query_historical_price(from_asset, to_asset, timestamp)
+
+    with patch('rotkehlchen.history.price.PriceHistorian.query_historical_price', side_effect=mock_query_historical_price):  # noqa: E501
+        response = requests.post(
+            api_url_for(
+                rotkehlchen_api_server,
+                'historicalpricesperassetresource',
+            ),
+            json={
+                'asset': 'BTC',
+                'from_timestamp': START_TS,
+                'to_timestamp': START_TS + 3 * DAY_IN_SECONDS,
+                'interval': DAY_IN_SECONDS,
+            },
+        )
+        result = assert_proper_sync_response_with_result(response)
+        # The first timestamp should have a price
+        assert str(START_TS) in result['prices']
+
+        # The second timestamp should be rate limited
+        assert str(START_TS + DAY_IN_SECONDS) not in result['prices']
+        assert START_TS + DAY_IN_SECONDS not in result['no_prices_timestamps']
path: rotkehlchen/tests/api/test_historical_balances.py
user: login: yabirgb
body: maybe create a variable for `tomorrow = START_TS + DAY_IN_SECONDS` since you use it a lot in the test. Actually you have it as future_ts
created_at: 2025-03-06T08:53:39Z
author_association: MEMBER
diff_hunk: @@ -14168,12 +14169,14 @@ Historical Balance Queries
               "1672704000": "19800"
             },
             "no_prices_timestamps": [1672790400, 1672876800]
+            "rate_limited_prices_timestamps": [1672790400, 1672876800]
path: docs/api.rst
user: login: yabirgb
body: ```suggestion
            "no_prices_timestamps": [1672790400, 1672876800],
            "rate_limited_prices_timestamps": [1672790400, 1672876800]
```
created_at: 2025-03-06T08:55:26Z
author_association: MEMBER
diff_hunk: @@ -826,3 +829,96 @@ def test_get_historical_prices_per_asset(
         str(START_TS): '10',
         str(START_TS + DAY_IN_SECONDS): '20',
     }
+
+    # test future timestamp exclusion
+    current_time = ts_now()
+    future_ts = current_time + DAY_IN_SECONDS
+    response = requests.post(
+        api_url_for(
+            rotkehlchen_api_server,
+            'historicalpricesperassetresource',
+        ),
+        json={
+            'asset': 'BTC',
+            'from_timestamp': current_time - DAY_IN_SECONDS,
+            'to_timestamp': future_ts,  # Include timestamp in the future
+            'interval': DAY_IN_SECONDS,
+        },
+    )
+    result = assert_proper_sync_response_with_result(response)
+    assert str(future_ts) not in result['prices']
path: rotkehlchen/tests/api/test_historical_balances.py
user: login: prettyirrelevant
body: it won't be exactly current_time since timestamps are generated at fixed intervals (from_timestamp + interval*n).
the maximum timestamp would be the highest interval boundary that doesn't exceed current_time, not current_time itself.

or did i misunderstand what you meant?
created_at: 2025-03-06T12:24:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -5299,25 +5304,30 @@ def get_historical_prices_per_asset(
             return _wrap_in_ok_result(result={
                 'prices': prices,
                 'no_prices_timestamps': no_prices_ts,
+                'rate_limited_prices_timestamps': rate_limited_prices_ts,
             })
 
+        total = len(timestamps)
path: rotkehlchen/api/rest.py
user: login: LefterisJP
body: why create a new variable for this?
created_at: 2025-03-06T12:53:46Z
author_association: MEMBER
diff_hunk: @@ -5299,25 +5304,30 @@ def get_historical_prices_per_asset(
             return _wrap_in_ok_result(result={
                 'prices': prices,
                 'no_prices_timestamps': no_prices_ts,
+                'rate_limited_prices_timestamps': rate_limited_prices_ts,
             })
 
+        total = len(timestamps)
path: rotkehlchen/api/rest.py
user: login: LefterisJP
body: forget it. it's a loop. That's why
created_at: 2025-03-06T13:17:48Z
author_association: MEMBER

PR comments:
user: login: codecov[bot]
created_at: 2025-03-05T21:03:03Z
body: ## [Codecov](https://app.codecov.io/gh/rotki/rotki/pull/9566?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 55.97%. Comparing base [(`f958c8a`)](https://app.codecov.io/gh/rotki/rotki/commit/f958c8a32f951ed0a280a15056b92d28208fa0f3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki) to head [(`f1c825a`)](https://app.codecov.io/gh/rotki/rotki/commit/f1c825ab78a351f6cfb323058fda86da5ed15e4a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki).
> Report is 3 commits behind head on develop.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           develop    #9566      +/-   ##
===========================================
+ Coverage    50.29%   55.97%   +5.68%     
===========================================
  Files         1891     1891              
  Lines       138911   138916       +5     
  Branches     13780    13781       +1     
===========================================
+ Hits         69861    77754    +7893     
+ Misses       66460    58350    -8110     
- Partials      2590     2812     +222     
```

| [Flag](https://app.codecov.io/gh/rotki/rotki/pull/9566/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki) | Coverage Δ | |
|---|---|---|
| [backend](https://app.codecov.io/gh/rotki/rotki/pull/9566/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki) | `81.00% <100.00%> (+12.57%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/rotki/rotki/pull/9566?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rotki).

<details><summary>🚀 New features to boost your workflow: </summary>

- ❄ [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- 📦 [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
author_association: NONE
user: login: prettyirrelevant
created_at: 2025-03-06T13:49:34Z
body: failing test due to cassettes now passes.
author_association: CONTRIBUTOR
user: login: rotkibot
created_at: 2025-03-06T13:49:57Z
body: rotki/test-caching#112 was successfully merged
author_association: NONE