REF/BENCH: tslibs-specific parts of asvs
State: closed

Motivation: asv runs take way too long.  One way to get around that is to make it easier to identify\* which benchmarks to run for a given PR.  To that extent, if we can identify benchmarks that don't rely on anything outside of tslibs\**, then a PR that doesn't touch that code can skip those benchmarks.  We can do the same thing for other low-dependency modules like _libs.algos.

\* Ideally in an automated way, but I'm not there yet.
\*\* Here and elsewhere, "tslibs" informally includes tseries.offsets and parts of tseries.frequences (until we can get all of that into the tslibs directory)

@TomAugspurger will moving benchmarks cause asv to start new time series, or will it successfully link up the new location to the old?

Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
Not sure, but I imagine that renaming will start a new timeseries.

I don't think that's a huge problem though (assuming this benchmark doesn't *introduce* a slowdown).
> Do you know what tests are taking the longest? I have an unverified suspicion that parametrized benchmarks like PeriodProperties might be taking longer than you would expect

I don't have a good a answer for that, but I can tell you

- I just did a `time asv continuous -E virtualenv -f 1.1 master HEAD -b algorithms` and it took 55 minutes.
- A few weeks ago I checked and found that a full asv run for a single commit had just over 5000 subprocesses import pandas.

Can you post the JSON result file? I think should have start / end time stamps in it

Sent from my iPhone

> On Oct 30, 2019, at 11:51 AM, jbrockmendel <notifications@github.com> wrote:
> 
> ﻿
> Do you know what tests are taking the longest? I have an unverified suspicion that parametrized benchmarks like PeriodProperties might be taking longer than you would expect
> 
> I don't have a good a answer for that, but I can tell you
> 
> I just did a time asv continuous -E virtualenv -f 1.1 master HEAD -b algorithms and it took 55 minutes.
> A few weeks ago I checked and found that a full asv run for a single commit had just over 5000 subprocesses import pandas.
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or unsubscribe.

> Can you post the JSON result file? I think should have start / end time stamps in it

[asv_log.txt](https://github.com/pandas-dev/pandas/files/3791368/asv_log.txt)  This was produced with `-b algorithms`, so not a full run.

For kicks I tried running asv continuous without selecting _any_ benchmarks, and with with environments already built, and it took 15 minutes
> For kicks I tried running asv continuous without selecting _any_ benchmarks, and with with environments already built, and it took 15 minutes

It would be interesting if you ran with the --verbose flag to see where all of that time was spent. Also do you have ccache installed?
> It would be interesting if you ran with the --verbose flag to see where all of that time was spent. 

Will give this a go.

> Also do you have ccache installed?

I don't know.  is this a pip thing or an apt/brew thing?  Is it relevant that I make heavy use of `git worktree`?
It is a homebrew thing
lgtm.
Yea lgtm as well. Can continue figuring out the ASV nuances outside of this
FWIW looks like ccache helped quite a bit on OSX but not on ubuntu (maybe needs config after install)
> For kicks I tried running asv continuous without selecting any benchmarks, and with with environments already built, and it took 15 minutes

Update: upgrading asv took this down to a few seconds.