Compare performance between Python versions

As #16 introduces Python 3 compatibility, it is interesting to see the performance of Django between Python versions. This PR builds on #16 and allows to define a control and experiment python executable.

This is the output of Python 2.7.5 vs Python 3.3.2 on current master:

    Control: Django 1.7.dev20131113053534 (in git branch master)
    Experiment: Django 1.7.dev20131113053534 (in git branch master)

    Running 'default_middleware' benchmark ...
    Min: -0.000719 -> 0.000426: -0.5922x slower
    Avg: 0.000566 -> 0.000624: 1.1020x slower
    Significant (t=-2.748613)
    Stddev: 0.00032 -> 0.00035: 1.1045x larger (N = 500)

    Running 'form_clean' benchmark ...
    Min: 0.000026 -> 0.000022: 1.1619x faster
    Avg: 0.000030 -> 0.000023: 1.2764x faster
    Significant (t=17.565230)
    Stddev: 0.00001 -> 0.00000: 3.4337x smaller (N = 500)

    Running 'form_create' benchmark ...
    Min: 0.000107 -> 0.000106: 1.0076x faster
    Avg: 0.000314 -> 0.000512: 1.6294x slower
    Not significant
    Stddev: 0.00449 -> 0.00897: 1.9975x larger (N = 500)

    Running 'l10n_render' benchmark ...
    Min: 0.008388 -> 0.008317: 1.0085x faster
    Avg: 0.009594 -> 0.009049: 1.0602x faster
    Significant (t=4.947870)
    Stddev: 0.00193 -> 0.00152: 1.2700x smaller (N = 500)

    Running 'model_creation' benchmark ...
    Min: 0.000209 -> 0.000246: 1.1776x slower
    Avg: 0.000225 -> 0.000268: 1.1917x slower
    Significant (t=-48.706113)
    Stddev: 0.00001 -> 0.00001: 1.0424x smaller (N = 500)

    Running 'model_delete' benchmark ...
    Min: 0.000281 -> 0.000310: 1.1036x slower
    Avg: 0.000312 -> 0.000324: 1.0362x slower
    Significant (t=-9.571304)
    Stddev: 0.00002 -> 0.00002: 1.1513x larger (N = 500)

    Running 'model_save_existing' benchmark ...
    Min: 0.012107 -> 0.012251: 1.0119x slower
    Avg: 0.012983 -> 0.013001: 1.0014x slower
    Not significant
    Stddev: 0.00155 -> 0.00085: 1.8180x smaller (N = 500)

    Running 'model_save_new' benchmark ...
    Min: 0.012386 -> 0.012633: 1.0199x slower
    Avg: 0.013068 -> 0.013415: 1.0266x slower
    Significant (t=-3.655738)
    Stddev: 0.00147 -> 0.00153: 1.0415x larger (N = 500)

    Running 'multi_value_dict' benchmark ...
    Min: 0.000058 -> 0.000055: 1.0521x faster
    Avg: 0.000591 -> 0.000478: 1.2364x faster
    Significant (t=6.291685)
    Stddev: 0.00032 -> 0.00025: 1.2918x smaller (N = 500)

    Running 'qs_filter_chaining' benchmark ...
    Min: 0.000808 -> 0.000855: 1.0577x slower
    Avg: 0.000855 -> 0.000875: 1.0240x slower
    Significant (t=-4.397596)
    Stddev: 0.00010 -> 0.00003: 3.3080x smaller (N = 500)

    Running 'query_aggregate' benchmark ...
    Min: 0.000234 -> 0.000257: 1.0986x slower
    Avg: 0.000245 -> 0.000265: 1.0815x slower
    Significant (t=-10.170246)
    Stddev: 0.00004 -> 0.00001: 3.8965x smaller (N = 500)

    Running 'query_all' benchmark ...
    Min: 0.028170 -> 0.030800: 1.0934x slower
    Avg: 0.030341 -> 0.033173: 1.0933x slower
    Significant (t=-12.495336)
    Stddev: 0.00318 -> 0.00394: 1.2392x larger (N = 500)

    Running 'query_all_multifield' benchmark ...
    Min: 0.057457 -> 0.060101: 1.0460x slower
    Avg: 0.060542 -> 0.063242: 1.0446x slower
    Significant (t=-10.213039)
    Stddev: 0.00382 -> 0.00451: 1.1827x larger (N = 500)

    Running 'query_annotate' benchmark ...
    Min: 0.000576 -> 0.000648: 1.1255x slower
    Avg: 0.000613 -> 0.000683: 1.1143x slower
    Significant (t=-14.584047)
    Stddev: 0.00010 -> 0.00004: 2.3773x smaller (N = 500)

    Running 'query_complex_filter' benchmark ...
    Min: 0.000122 -> 0.000128: 1.0507x slower
    Avg: 0.000127 -> 0.000133: 1.0454x slower
    Significant (t=-13.373395)
    Stddev: 0.00001 -> 0.00001: 1.0922x smaller (N = 500)

    Running 'query_count' benchmark ...
    Min: 0.000172 -> 0.000187: 1.0869x slower
    Avg: 0.000180 -> 0.000192: 1.0691x slower
    Significant (t=-12.841720)
    Stddev: 0.00002 -> 0.00001: 2.4602x smaller (N = 500)

    Running 'query_dates' benchmark ...
    Min: 0.000521 -> 0.000504: 1.0343x faster
    Avg: 0.000553 -> 0.000522: 1.0594x faster
    Significant (t=12.506188)
    Stddev: 0.00005 -> 0.00003: 1.5757x smaller (N = 500)

    Running 'query_delete' benchmark ...
    Min: 0.000213 -> 0.000259: 1.2183x slower
    Avg: 0.000225 -> 0.000266: 1.1793x slower
    Significant (t=-42.986316)
    Stddev: 0.00002 -> 0.00001: 1.2069x smaller (N = 500)

    Running 'query_delete_related' benchmark ...
    Min: 0.000303 -> 0.000335: 1.1059x slower
    Avg: 0.000318 -> 0.000347: 1.0917x slower
    Significant (t=-6.652992)
    Stddev: 0.00008 -> 0.00006: 1.2277x smaller (N = 500)

    Running 'query_distinct' benchmark ...
    Min: 0.000299 -> 0.000324: 1.0830x slower
    Avg: 0.000311 -> 0.000338: 1.0862x slower
    Significant (t=-22.202815)
    Stddev: 0.00002 -> 0.00002: 1.2534x larger (N = 500)

    Running 'query_exclude' benchmark ...
    Min: 0.000449 -> 0.000482: 1.0727x slower
    Avg: 0.000470 -> 0.000526: 1.1173x slower
    Significant (t=-14.053203)
    Stddev: 0.00002 -> 0.00009: 4.2894x larger (N = 500)

    Running 'query_exists' benchmark ...
    Min: 0.000787 -> 0.000818: 1.0394x slower
    Avg: 0.000824 -> 0.000835: 1.0135x slower
    Significant (t=-3.370657)
    Stddev: 0.00007 -> 0.00002: 3.0109x smaller (N = 500)

    Running 'query_filter' benchmark ...
    Min: 0.000357 -> 0.000380: 1.0639x slower
    Avg: 0.000371 -> 0.000428: 1.1546x slower
    Significant (t=-17.188760)
    Stddev: 0.00002 -> 0.00007: 3.9123x larger (N = 500)

    Running 'query_get' benchmark ...
    Min: 0.024963 -> 0.025388: 1.0170x slower
    Avg: 0.026989 -> 0.026812: 1.0066x faster
    Not significant
    Stddev: 0.00222 -> 0.00213: 1.0427x smaller (N = 500)

    Running 'query_get_or_create' benchmark ...
    Min: 0.001117 -> 0.001115: 1.0015x faster
    Avg: 0.001196 -> 0.001193: 1.0025x faster
    Not significant
    Stddev: 0.00006 -> 0.00005: 1.2736x smaller (N = 500)

    Running 'query_in_bulk' benchmark ...
    Min: 0.000422 -> 0.000443: 1.0493x slower
    Avg: 0.000459 -> 0.000473: 1.0288x slower
    Significant (t=-3.127283)
    Stddev: 0.00007 -> 0.00006: 1.1924x smaller (N = 500)

    Running 'query_iterator' benchmark ...
    Min: 0.000259 -> 0.000288: 1.1101x slower
    Avg: 0.000283 -> 0.000303: 1.0711x slower
    Significant (t=-8.740333)
    Stddev: 0.00004 -> 0.00003: 1.3828x smaller (N = 500)

    Running 'query_latest' benchmark ...
    Min: 0.000338 -> 0.000366: 1.0819x slower
    Avg: 0.000363 -> 0.000379: 1.0443x slower
    Significant (t=-14.680610)
    Stddev: 0.00002 -> 0.00002: 1.0729x smaller (N = 500)

    Running 'query_none' benchmark ...
    Min: 0.000134 -> 0.000133: 1.0042x faster
    Avg: 0.000142 -> 0.000143: 1.0096x slower
    Significant (t=-2.455846)
    Stddev: 0.00001 -> 0.00001: 1.0503x larger (N = 500)

    Running 'query_order_by' benchmark ...
    Min: 0.000346 -> 0.000368: 1.0639x slower
    Avg: 0.000399 -> 0.000383: 1.0406x faster
    Significant (t=3.137543)
    Stddev: 0.00011 -> 0.00002: 6.5895x smaller (N = 500)

    Running 'query_prefetch_related' benchmark ...
    Min: 0.092059 -> 0.094962: 1.0315x slower
    Avg: 0.100182 -> 0.103258: 1.0307x slower
    Significant (t=-7.117510)
    Stddev: 0.00684 -> 0.00683: 1.0009x smaller (N = 500)

    Running 'query_raw' benchmark ...
    Min: 0.021337 -> 0.022009: 1.0315x slower
    Avg: 0.023503 -> 0.023489: 1.0006x faster
    Not significant
    Stddev: 0.00222 -> 0.00272: 1.2261x larger (N = 500)

    Running 'query_raw_deferred' benchmark ...
    Min: 0.021762 -> 0.019842: 1.0968x faster
    Avg: 0.023257 -> 0.021674: 1.0730x faster
    Significant (t=11.028396)
    Stddev: 0.00195 -> 0.00255: 1.3036x larger (N = 500)

    Running 'query_select_related' benchmark ...
    Min: 0.059516 -> 0.061595: 1.0349x slower
    Avg: 0.063593 -> 0.066907: 1.0521x slower
    Significant (t=-11.991853)
    Stddev: 0.00365 -> 0.00498: 1.3653x larger (N = 500)

    Running 'query_update' benchmark ...
    Min: 0.000194 -> 0.000215: 1.1084x slower
    Avg: 0.000219 -> 0.000232: 1.0583x slower
    Significant (t=-10.316140)
    Stddev: 0.00003 -> 0.00001: 2.3351x smaller (N = 500)

    Running 'query_values' benchmark ...
    Min: 0.000274 -> 0.000293: 1.0710x slower
    Avg: 0.000324 -> 0.000324: 1.0017x slower
    Not significant
    Stddev: 0.00009 -> 0.00003: 2.5640x smaller (N = 500)

    Running 'query_values_list' benchmark ...
    Min: 0.000260 -> 0.000284: 1.0944x slower
    Avg: 0.000280 -> 0.000317: 1.1317x slower
    Significant (t=-13.653464)
    Stddev: 0.00005 -> 0.00004: 1.2393x smaller (N = 500)

    Running 'raw_sql' benchmark ...
    Min: 0.000048 -> 0.000056: 1.1706x slower
    Avg: 0.000050 -> 0.000061: 1.2245x slower
    Significant (t=-26.829219)
    Stddev: 0.00000 -> 0.00001: 2.6213x larger (N = 500)

    Running 'startup' benchmark ...
    0.143236 -> 0.204983: 1.4311x slower

    Running 'template_compilation' benchmark ...
    Min: 0.000635 -> 0.000668: 1.0524x slower
    Avg: 0.000731 -> 0.000821: 1.1226x slower
    Not significant
    Stddev: 0.00137 -> 0.00212: 1.5406x larger (N = 500)

    Running 'template_render' benchmark ...
    Min: 0.012701 -> 0.013381: 1.0536x slower
    Avg: 0.013860 -> 0.014345: 1.0350x slower
    Significant (t=-2.688239)
    Stddev: 0.00232 -> 0.00331: 1.4271x larger (N = 500)

    Running 'template_render_simple' benchmark ...
    Min: 0.000108 -> 0.000116: 1.0696x slower
    Avg: 0.000175 -> 0.000232: 1.3217x slower
    Not significant
    Stddev: 0.00135 -> 0.00234: 1.7419x larger (N = 500)

    Running 'url_resolve' benchmark ...
    Min: 0.019546 -> 0.015493: 1.2616x faster
    Avg: 0.021172 -> 0.016535: 1.2805x faster
    Significant (t=43.380995)
    Stddev: 0.00169 -> 0.00169: 1.0034x smaller (N = 500)

    Running 'url_resolve_flat' benchmark ...
    Min: 0.295093 -> 0.246289: 1.1982x faster
    Avg: 0.314733 -> 0.263176: 1.1959x faster
    Significant (t=60.861577)
    Stddev: 0.01485 -> 0.01176: 1.2633x smaller (N = 500)

    Running 'url_resolve_flat_i18n_off' benchmark ...
    Min: 0.403079 -> 0.365110: 1.1040x faster
    Avg: 0.428826 -> 0.392236: 1.0933x faster
    Significant (t=26.168553)
    Stddev: 0.02376 -> 0.02033: 1.1686x smaller (N = 500)

    Running 'url_resolve_nested' benchmark ...
    Min: 0.000115 -> 0.000116: 1.0123x slower
    Avg: 0.000151 -> 0.000161: 1.0663x slower
    Not significant
    Stddev: 0.00056 -> 0.00053: 1.0625x smaller (N = 500)

    Running 'url_reverse' benchmark ...
    Min: 0.000320 -> 0.000321: 1.0021x slower
    Avg: 0.000350 -> 0.000343: 1.0184x faster
    Not significant
    Stddev: 0.00016 -> 0.00018: 1.1258x larger (N = 500)