Repository: QuantConnect/Lean

Issue #3061: SetupCurrencyConversions asks for history data ahead of Now time
Created: 2019-04-08T00:05:58Z
Expected Behavior
Cannot ask for history ahead of now time
Actual Behavior

Fails to start Lean, because start date is ahead of now date, failing with the following error
20190408 01:54:25.790 ERROR:: Engine.Run(): During the algorithm initialization, the following exception has occurred: Oanda.RestV20.Client.ApiException: Error calling GetInstrumentCandles: {"errorMessage":"Invalid value specified for 'from'. Time is in the future"} at Oanda.RestV20.Api.DefaultApi.GetInstrumentCandlesWithHttpInfo (System.String authorization, System.String instrument, System.String acceptDatetimeFormat, System.String price, System.String granularity, System.Nullable1[T] count, System.String from, System.String to, System.Nullable1[T] smooth, System.Nullable1[T] includeFirst, System.Nullable1[T] dailyAlignment, System.String alignmentTimezone, System.String weeklyAlignment) [0x00362] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Brokerages/Oanda/RestV20/Api/DefaultApi.cs:3816  at Oanda.RestV20.Api.DefaultApi.GetInstrumentCandles (System.String authorization, System.String instrument, System.String acceptDatetimeFormat, System.String price, System.String granularity, System.Nullable1[T] count, System.String from, System.String to, System.Nullable1[T] smooth, System.Nullable1[T] includeFirst, System.Nullable1[T] dailyAlignment, System.String alignmentTimezone, System.String weeklyAlignment) [0x00001] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Brokerages/Oanda/RestV20/Api/DefaultApi.cs:3735  at QuantConnect.Brokerages.Oanda.OandaRestApiV20+<DownloadQuoteBars>d__25.MoveNext () [0x00084] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Brokerages/Oanda/OandaRestApiV20.cs:566  at System.Collections.Generic.List1[T].AddEnumerable (System.Collections.Generic.IEnumerable1[T] enumerable) [0x00059] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs:1108  at System.Collections.Generic.List1[T]..ctor (System.Collections.Generic.IEnumerable1[T] collection) [0x00062] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs:87  at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable1[T] source) [0x0000e] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/external/corefx/src/System.Linq/src/System/Linq/ToCollection.cs:30
at QuantConnect.Brokerages.Oanda.OandaBrokerage+d__14.MoveNext () [0x000f6] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Brokerages/Oanda/OandaBrokerage.cs:228
at QuantConnect.Lean.Engine.DataFeeds.Enumerators.QuoteBarFillForwardEnumerator.MoveNext () [0x00001] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Engine/DataFeeds/Enumerators/QuoteBarFillForwardEnumerator.cs:70
at QuantConnect.Lean.Engine.DataFeeds.Enumerators.FillForwardEnumerator.MoveNext () [0x00088] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Engine/DataFeeds/Enumerators/FillForwardEnumerator.cs:137
at QuantConnect.Lean.Engine.DataFeeds.SubscriptionData+d__8.MoveNext () [0x00066] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Engine/DataFeeds/SubscriptionData.cs:80
at QuantConnect.Lean.Engine.DataFeeds.Subscription.MoveNext () [0x00010] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Engine/DataFeeds/Subscription.cs:201
at QuantConnect.Lean.Engine.HistoricalData.BrokerageHistoryProvider.GetHistory (System.Collections.Generic.IEnumerable1[T] requests, NodaTime.DateTimeZone sliceTimeZone) [0x0002f] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Engine/HistoricalData/BrokerageHistoryProvider.cs:71  at QuantConnect.Lean.Engine.Setup.BaseSetupHandler.SetupCurrencyConversions (QuantConnect.Interfaces.IAlgorithm algorithm, QuantConnect.Lean.Engine.DataFeeds.UniverseSelection universeSelection) [0x001b7] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Engine/Setup/BaseSetupHandler.cs:92  at QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler.Setup (QuantConnect.Lean.Engine.Setup.SetupHandlerParameters parameters) [0x005ac] in /Users/erikbengtson/Documents/Projects/erikbengtson/Lean/Engine/Setup/BrokerageSetupHandler.cs:354 Error calling GetInstrumentCandles: {"errorMessage":"Invalid value specified for 'from'. Time is in the future"} 
Potential Solution

Reproducing the Problem

Start lean on Sunday night (before monday 12am utc). If you start lean on monday 12am, then problem does not occur
System Information

macOS
Checklist



 I have completely filled out this template
 I have confirmed that this issue exists on the current master branch
 I have confirmed that this is not a duplicate issue by searching issues



 I have provided detailed steps to reproduce the issue

Duplicate issue #4849: Lean is Live Mode start date is ignoring the Time, which causes issues when launching Lean after midnight on timezone GMT+2
Created: 2020-10-12T22:55:58Z
Expected Behavior
Live mode startdate contains date but lacks time and leads to incorrect datetime
Actual Behavior
time is ignored, and thus when converting the date, it leads to a future date time
Potential Solution
SetLiveMode should use DateTime.UtcNow rather than using DateTime.Today
in QCAlgorithm, change SetLiveMode to this:
        public void SetLiveMode(bool live) { if (!_locked) { _liveMode = live; Notify = new NotificationManager(live); TradeBuilder.SetLiveMode(live); Securities.SetLiveMode(live); if (live) { _startDate = DateTime.UtcNow; _endDate = QuantConnect.Time.EndOfTime; } } }
Reproducing the Problem
Launch Lean in live Mode under this configuration:
machine is using UTC+2, lean is launch between midnight and 1am in the morning.
Using oanda brokerage. Lean will fail with the below error:
20201013 00:50:18.973 Trace:: Event Name "EveryDay: Every 10 min", scheduled to run at 10/12/2020 12:00:00 AM (UTC)... 20201013 00:50:18.990 Trace:: LiveTradingResultHandler.SendStatusUpdate(): status: 'LoggingIn'.  Logging into brokerage... 20201013 00:50:18.991 Trace:: BrokerageSetupHandler.Setup(): Connecting to brokerage... 20201013 00:50:18.991 Trace:: BrokerageSetupHandler.Setup(): Fetching cash balance from brokerage... Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.Web/4.0.0.0__b03f5f7f11d50a3a/System.Web.dll [External] Thread started: <Thread Pool> #18 20201013 00:50:19.507 Trace:: BrokerageSetupHandler.Setup(): Setting EUR cash to 10176.2068 20201013 00:50:19.510 Trace:: BrokerageSetupHandler.Setup(): Fetching open orders from brokerage... 20201013 00:50:19.648 Trace:: BrokerageSetupHandler.Setup(): Fetching holdings from brokerage... 20201013 00:50:19.910 Trace:: LiveTradingDataFeed.CreateUniverseSubscription(): Creating user defined universe: MANUAL-UNIVERSE-SELECTION-MODEL-FOREX-OANDA--2001353904 8G Loaded assembly: Anonymously Hosted DynamicMethods Assembly [External] 20201013 00:50:19.994 Trace:: DataManager.AddSubscription(): Added MANUAL-UNIVERSE-SELECTION-MODEL-FOREX-OANDA--2001353904,MANUAL-UNIVERSE-SELECTION-MODEL-FOREX-OANDA--2001353904,Minute,TradeBar,Trade,Adjusted,Internal. Start: 10/13/2020 12:00:00 AM. End: 12/31/2050 12:00:00 AM 20201013 00:50:19.997 Trace:: LiveTradingDataFeed.CreateUniverseSubscription(): Creating user defined universe: MANUAL-UNIVERSE-SELECTION-MODEL-CFD-OANDA-499175433 8I 20201013 00:50:19.997 Trace:: DataManager.AddSubscription(): Added MANUAL-UNIVERSE-SELECTION-MODEL-CFD-OANDA-499175433,MANUAL-UNIVERSE-SELECTION-MODEL-CFD-OANDA-499175433,Minute,TradeBar,Trade,Adjusted,Internal. Start: 10/13/2020 12:00:00 AM. End: 12/31/2050 12:00:00 AM 20201013 00:50:20.051 Trace:: Cash.EnsureCurrencyDataFeed(): Adding EURUSD for cash EUR currency feed Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/Microsoft.CSharp/4.0.0.0__b03f5f7f11d50a3a/Microsoft.CSharp.dll [External] 20201013 00:50:20.378 ERROR:: FitnessScore.Initialize(): fitness score will not be calculated because the algorithms starting portfolio value is 0. 20201013 00:50:20.399 ERROR:: LiveTradingResultHandler.Update(): Algorithm not yet initialized. Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.Transactions/4.0.0.0__b77a5c561934e089/System.Transactions.dll [External] Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.ServiceModel.Internals/0.0.0.0__b77a5c561934e089/System.ServiceModel.Internals.dll [External] Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/Mono.WebBrowser/4.0.0.0__0738eb9f132ed756/Mono.WebBrowser.dll [External] Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.IdentityModel/4.0.0.0__b77a5c561934e089/System.IdentityModel.dll [External] Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.Security/4.0.0.0__b03f5f7f11d50a3a/System.Security.dll [External] Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.Web.ApplicationServices/4.0.0.0__31bf3856ad364e35/System.Web.ApplicationServices.dll [External] Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.Messaging/4.0.0.0__b03f5f7f11d50a3a/System.Messaging.dll [External] Loaded assembly: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/gac/System.Web.Services/4.0.0.0__b03f5f7f11d50a3a/System.Web.Services.dll [External] 20201013 00:50:20.983 ERROR:: Engine.Run(): During the algorithm initialization, the following exception has occurred: Oanda.RestV20.Client.ApiException: Error calling GetInstrumentCandles: {"errorMessage":"Invalid value specified for 'from'. Time is in the future"} at Oanda.RestV20.Api.DefaultApi.GetInstrumentCandlesWithHttpInfo (System.String authorization, System.String instrument, System.String acceptDatetimeFormat, System.String price, System.String granularity, System.Nullable1[T] count, System.String from, System.String to, System.Nullable1[T] smooth, System.Nullable1[T] includeFirst, System.Nullable1[T] dailyAlignment, System.String alignmentTimezone, System.String weeklyAlignment) [0x00362] in /Lean-master-2/Brokerages/Oanda/RestV20/Api/DefaultApi.cs:3816  at Oanda.RestV20.Api.DefaultApi.GetInstrumentCandles (System.String authorization, System.String instrument, System.String acceptDatetimeFormat, System.String price, System.String granularity, System.Nullable1[T] count, System.String from, System.String to, System.Nullable1[T] smooth, System.Nullable1[T] includeFirst, System.Nullable1[T] dailyAlignment, System.String alignmentTimezone, System.String weeklyAlignment) [0x00001] in /Lean-master-2/Brokerages/Oanda/RestV20/Api/DefaultApi.cs:3735  at QuantConnect.Brokerages.Oanda.OandaRestApiV20+<DownloadQuoteBars>d__25.MoveNext () [0x00089] in /Lean-master-2/Brokerages/Oanda/OandaRestApiV20.cs:561  at System.Collections.Generic.List1[T].AddEnumerable (System.Collections.Generic.IEnumerable1[T] enumerable) [0x00059] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs:1108  at System.Collections.Generic.List1[T]..ctor (System.Collections.Generic.IEnumerable1[T] collection) [0x00062] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs:87  at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable1[T] source) [0x0000e] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corefx/src/System.Linq/src/System/Linq/ToCollection.cs:30
at QuantConnect.Brokerages.Oanda.OandaBrokerage+d__14.MoveNext () [0x000f6] in /Lean-master-2/Brokerages/Oanda/OandaBrokerage.cs:230
at QuantConnect.Lean.Engine.DataFeeds.Enumerators.QuoteBarFillForwardEnumerator.MoveNext () [0x00001] in /Lean-master-2/Engine/DataFeeds/Enumerators/QuoteBarFillForwardEnumerator.cs:70
at QuantConnect.Lean.Engine.DataFeeds.Enumerators.FillForwardEnumerator.MoveNext () [0x00088] in /Lean-master-2/Engine/DataFeeds/Enumerators/FillForwardEnumerator.cs:147
at QuantConnect.Lean.Engine.DataFeeds.Enumerators.SubscriptionDataEnumerator.MoveNext () [0x00001] in /Lean-master-2/Engine/DataFeeds/Enumerators/SubscriptionDataEnumerator.cs:67
at QuantConnect.Lean.Engine.DataFeeds.Subscription.MoveNext () [0x00010] in /Lean-master-2/Engine/DataFeeds/Subscription.cs:206
at QuantConnect.Lean.Engine.HistoricalData.SynchronizingHistoryProvider+d__3.MoveNext () [0x000ac] in /Lean-master-2/Engine/HistoricalData/SynchronizingHistoryProvider.cs:59
at QuantConnect.Data.SliceExtensions.PushThrough (System.Collections.Generic.IEnumerable1[T] slices, System.Action1[T] handler) [0x000cd] in /Lean-master-2/Common/Data/SliceExtensions.cs:209
at QuantConnect.Lean.Engine.Setup.BaseSetupHandler.SetupCurrencyConversions (QuantConnect.Interfaces.IAlgorithm algorithm, QuantConnect.Lean.Engine.DataFeeds.UniverseSelection universeSelection) [0x00168]
20201013 00:50:20.985 Trace:: JOB HANDLERS:
20201013 00:50:20.985 Trace::          DataFeed:     QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed
20201013 00:50:20.985 Trace::          Setup:        QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler
20201013 00:50:20.985 Trace::          RealTime:     QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler
20201013 00:50:20.985 Trace::          Results:      QuantConnect.Lean.Engine.Results.LiveTradingResultHandler
20201013 00:50:20.985 Trace::          Transactions: QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler
20201013 00:50:20.985 Trace::          Alpha:        QuantConnect.Lean.Engine.Alphas.DefaultAlphaHandler
20201013 00:50:20.985 Trace::          ObjectStore:  QuantConnect.Lean.Engine.Storage.LocalObjectStore
20201013 00:50:20.986 Trace::          History Provider:     QuantConnect.Lean.Engine.HistoricalData.BrokerageHistoryProvider
20201013 00:50:20.986 Trace::          Brokerage:      QuantConnect.Brokerages.Oanda.OandaBrokerage
20201013 00:50:20.986 Trace:: LiveTradingDataFeed.Exit(): Start. Setting cancellation token...
20201013 00:50:20.986 Trace:: BaseDataExchange(CustomDataExchange) Stopping...
20201013 00:50:20.987 Trace:: LiveTradingDataFeed.Exit(): Exit Finished.
20201013 00:50:20.987 Trace:: DefaultAlphaHandler.Exit(): Exiting...
20201013 00:50:20.994 Trace:: BaseDataExchange(CustomDataExchange).ConsumeQueue(): Cancellation requested.  Exiting...
Thread finished:  #15
20201013 00:50:20.995 Trace:: DefaultAlphaHandler.Exit(): Ended
`
System Information

Mac OS
Lean Latest
Checklist



 I have completely filled out this template
 I have confirmed that this issue exists on the current master branch
 I have confirmed that this is not a duplicate issue by searching issues



 I have provided detailed steps to reproduce the issue

Closing comment (2020-10-13T00:02:12Z): Hey @ebengtso, thanks for the detailed report, we will look into it, I'll close it since I believe it's a duplicate of #3061