DEPR: deprecating series asof GH10343
State: closed

closes #10343 and discussion on #10266 - deprecating `Series.asof(where)` in favour of `Series.reindex(where, method='ffill')`.


Labels:
name: Docs
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Deprecate
description: Functionality to remove in pandas

Comments:
this is ok, but I think this could use a small doc-section in `missing.rst` in the `fillna` subsection.

also will have to add a reference in the deprecate issue #6581 (will do this upon merging)

@jreback I'd be happy to add it, but I'm not entirely sure what you're looking for here. I had a look through the missing_data.rst and there's no mention of asof to correct/update.  Since it is a reindexing method, it's not clear how it should documented in the missing data section.

@bwillers can you rebase?

@jreback rebased and green

Can you put a mini-doc section right after here: http://pandas.pydata.org/pandas-docs/stable/timeseries.html#filling-forward-backward

showing an `asof` usecase (using the reindexing and fill method). Also a pointer in the whatsnew to this new doc section would be great. thxs.

If everyone thinks this is a good idea to deprecate, then so be it. I should say that we use this a lot, and find it a really convenient method. There are lots of use cases where you want to ask "what was the most recent valid value". And `asof` is much simpler than `Series.reindex(where, method='ffill')`, particularly when reading code.

Am I missing something? Or are we just particular in our use case?

@maximilianr Well, the motivation here is mainly based on `asof` being a re-implementation of functionality that already exists in the better tested and more widely applicable `reindex`. I absolutely agree that the basic operation of wanting to know the most recent value is a common one, and that `s.dropna().reindex(where, method='ffill')` is unfortunately a lot more verbose than `s.asof(where)`.

That said, `reindex` has the advantage of working for a `NDFrame` or `DataFrame`, and working with a `MultiIndex`, none of which are supported by `asof`.

The question becomes if the convenience of the shorter form for the specialized use case is worth the code bloat from multiple implementations of the same thing. I'm not personally convinced that it does, but that's definitely something worth discussing.

@bwillers completely agree with that synthesis - i.e. the big question being how useful a method needs to be in order to compensate for the additional clutter.

One point to consider for the specific case is how obvious that translation is for the average user. I'd have thought 'not very obvious', but I don't have a great handle on what the average current / future user looks like.

If we do keep it, having it as a simple wrapper of the translation seems wise.

Over to the maintainers on the broader question...

As a reference, `asof` is used in Python for Data Analysis

Note: I never used asof myself, so difficult to assess. 
But, if `asof` is used and is useful, I think we should reconsider the deprecation. I agree that the equivalent of `s.dropna().reindex(where, method='ffill')` is not that straightforward, and easy to come up with.

We either deprecate it, or we keep it and see this as an opportunitiy to:
- reimplement it using the existing and better tested reindexing machinery
- in this way, make the function more general (not restricted to timeseries)
- in the docstring, make it clear that this is a shortcut for a reindexing operation, and if you want more options (other filling etc) or use it for a dataframe you should use that. In this way, the function can also be somewhat educational to explain what reindexing does
- also explain this in the tutorial docs

And maybe it would be useful to have some kind of `skipna` kwarg in `reindex`?

@jorisvandenbossche That's not a bad idea - effectively 'replumb' the asof internals into reindex, but leave the compact and fairly intuitive api the same.

@bwillers ok!

ok, I revised the original issue. Let's try to reimplement this using `.reindex` internally, add tests, and leave the API.

@bwillers you can repurpose this PR or open a new one up to u

@jreback sounds good, running around like a headless chicken this week but hope to get this done next.