Repository: quantopian/zipline

Issue #2040: Intraday performance report when using 1 minute data?
Created: 2017-12-04T23:29:00Z
Q0017.txt
Dear Zipline Maintainers,
Before I tell you about my issue, let me describe my environment:
Environment

Operating System: (Windows Version or $ uname --all)
4.10.0-40-generic #44~16.04.1-Ubuntu SMP Thu Nov 9 15:37:44 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux * Python Version: $ python --version
Python 3.5.2
Python Bitness: $ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))'
64
How did you install Zipline: (pip, conda, or other (please explain))
pip
Python packages: $ pip freeze or $ conda list

alembic==0.9.6
bcolz==0.12.1
Bottleneck==1.2.1
certifi==2017.11.5
chardet==3.0.4
click==6.7
contextlib2==0.5.5
cycler==0.10.0
cyordereddict==1.0.0
Cython==0.27.3
decorator==4.1.2
empyrical==0.3.3
idna==2.6
intervaltree==2.1.0
Logbook==1.1.0
lru-dict==1.1.6
Mako==1.0.7
MarkupSafe==1.0
matplotlib==2.1.0
multipledispatch==0.4.9
networkx==2.0
numexpr==2.6.4
numpy==1.13.3
pandas==0.18.1
pandas-datareader==0.5.0
patsy==0.4.1
pyparsing==2.2.0
python-dateutil==2.6.1
python-editor==1.0.3
pytz==2017.3
requests==2.18.4
requests-file==1.4.2
requests-ftp==0.3.1
scipy==1.0.0
six==1.11.0
sortedcontainers==1.5.7
SQLAlchemy==1.1.15
statsmodels==0.8.0
tables==3.4.2
toolz==0.8.2
urllib3==1.22
zipline==1.1.1


Now that you know a little about me, let me tell you about the issue I am
having:
Description of Issue

What did you expect to happen?
Intraday performance report
What happened instead?
Daily performance report

The ingestion step works fine:
zipline ingest -b ingester
entering machina.  tuSymbols= ('Q0017',)
about to return ingest function
entering ingest and creating blank dfMetadata
dfMetadata <class 'pandas.core.frame.DataFrame'>
<bound method NDFrame.describe of   start_date   end_date auto_close_date symbol
0 1970-01-01 1970-01-01      1970-01-01   None>
S= Q0017 IFIL=/merged_data/Q0017.csv
read_csv dfData <class 'pandas.core.frame.DataFrame'> length 7717 2017-06-15 22:00:00
start_date <class 'pandas.tslib.Timestamp'> 2017-06-15 22:00:00 None
end_date <class 'pandas.tslib.Timestamp'> 2017-06-23 20:00:00 None
ac_date <class 'pandas.tslib.Timestamp'> 2017-06-24 20:00:00 None
liData <class 'list'> length 1
Now calling minute_bar_writer
returned from minute_bar_writer
calling asset_db_writer
dfMetadata <class 'pandas.core.frame.DataFrame'>
           start_date            end_date     auto_close_date symbol exchange
0 2017-06-15 22:00:00 2017-06-23 20:00:00 2017-06-24 20:00:00  Q0017      ICE
symbol_map <class 'pandas.core.series.Series'>
returned from asset_db_writer
calling adjustment_writer
returned from adjustment_writer
now leaving ingest function


So I try to run this toy example:

from zipline.api import symbol, get_datetime, record, order_target, history
from pytz import timezone



def initialize(context):
    context.contract  = symbol("Q0017")
    context.i         = 0
    
    
def handle_data(context, data):
    context.i += 1
    if context.i < 30:
        return

    # Compute averages
    # history() has to be called with the same params
    # from above and returns a pandas dataframe.
    short_mavg = data.history(context.contract, 'close', 10, '1m').mean()
    long_mavg  = data.history(context.contract, 'close', 30, '1m').mean()

    # Trading logic
    if short_mavg > long_mavg:
        # order_target orders as many shares as needed to
        # achieve the desired number of shares.
        order_target(context.contract, 100)
    elif short_mavg < long_mavg:
        order_target(context.contract, 0)

    # Save values for later inspection
    record(Q0017      = data.current(context.contract, "close"),
           short_mavg = short_mavg,
           long_mavg  = long_mavg)




# Note: this function can be removed if running
# this algorithm on quantopian.com
def analyze(context=None, results=None):
    import matplotlib.pyplot as plt
    import logbook
    logbook.StderrHandler().push_application()
    log = logbook.Logger('Algorithm')

    fig = plt.figure()
    ax1 = fig.add_subplot(211)
    results.portfolio_value.plot(ax=ax1)
    ax1.set_ylabel('Portfolio value (USD)')

    ax2 = fig.add_subplot(212)
    ax2.set_ylabel('Price (USD)')

    # If data has been record()ed, then plot it.
    # Otherwise, log the fact that no data has been recorded.
    if ('Q0017' in results and 'short_mavg' in results and
            'long_mavg' in results):
        results['Q0017'].plot(ax=ax2)
        results[['short_mavg', 'long_mavg']].plot(ax=ax2)

        trans = results.ix[[t != [] for t in results.transactions]]
        buys = trans.ix[[t[0]['amount'] > 0 for t in
                         trans.transactions]]
        sells = trans.ix[
            [t[0]['amount'] < 0 for t in trans.transactions]]
        ax2.plot(buys.index, results.short_mavg.ix[buys.index],
                 '^', markersize=10, color='m')
        ax2.plot(sells.index, results.short_mavg.ix[sells.index],
                 'v', markersize=10, color='k')
        plt.legend(loc=0)
    else:
        msg = 'Q0017, short_mavg & long_mavg data not captured using record().'
        ax2.annotate(msg, xy=(0.1, 0.5))
        log.info(msg)

    plt.show()

And it runs. but the performance report:

zipline run -f my_first_backtest.py --bundle ingester --data-frequency minute -s 2017-06-15 -e 2017-06-23
entering machina.  tuSymbols= ('Q0017',)
about to return ingest function
[2017-12-04 23:31:30.040649] WARNING: Loader: Refusing to download new benchmark data because a download succeeded at 2017-12-04 23:19:05.652054+00:00.
[2017-12-04 23:31:33.389892] INFO: Performance: Simulated 7 trading days out of 7.
[2017-12-04 23:31:33.390008] INFO: Performance: first open: 2017-06-15 13:31:00+00:00
[2017-12-04 23:31:33.390093] INFO: Performance: last close: 2017-06-23 20:00:00+00:00
                           Q0017  algo_volatility  algorithm_period_return  \
2017-06-15 20:00:00+00:00    NaN              NaN                 0.000000   
2017-06-16 20:00:00+00:00  47.29         0.000104                -0.000009   
2017-06-19 20:00:00+00:00  46.81         0.002406                -0.000276   
2017-06-20 20:00:00+00:00  45.84         0.006135                -0.001100   
2017-06-21 20:00:00+00:00  44.78         0.012046                -0.002896   
2017-06-22 20:00:00+00:00  45.29         0.013814                -0.002144   
2017-06-23 20:00:00+00:00  45.61         0.012915                -0.002037

has only 7 rows (one per day). Given 7717 intraday bars, I would expect
to see one row per bar (7717 of them).

Duplicate issue #2491: Perf results for minute frequency
Created: 2019-07-01T10:07:13Z
Hi @freddiev4 ,
When we use minute data the performance results are coming in daily basis.
Could you please help how can we get the perf result on minute basis?

Closing comment (2019-09-09T09:01:16Z): Duplicate of #2040 ?