BUG: Fix for .str.replace with repl function
State: closed

.str.replace now accepts a callable (function) as replacement string. It now raises a TypeError when repl is not string like nor a callable. Docstring updated accordingly.

  - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry
- [x] closes #15055


Labels:
name: Bug
name: Strings
description: String extension data type and string data

Comments:
Looks good, can you add tests and a whatsnew entry?
Thanks, @chris-b1 and @jreback. Where should I put the whatsnew entry? Can I just pick an empty spot in doc/source/whatsnew/v0.20.0.txt?

    - Bug in ``.str.replace`` now accepts a callable replacement which is passed to ``re.sub`` (:issue:`15055`)

This maybe a new feature or improvement rather than a bug, right?
Yeah, I think new features is fine - I had marked the issue a  bug because this would have worked in older versions, although basically undocumented.
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15056?src=pr) is 85.98% (diff: 100%)
> Merging [#15056](https://codecov.io/gh/pandas-dev/pandas/pull/15056?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **1.22%**

```diff
@@             master     #15056   diff @@
==========================================
  Files           145        140     -5   
  Lines         51146      51261   +115   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43351      44075   +724   
+ Misses         7795       7186   -609   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [018414e...90779ce](https://codecov.io/gh/pandas-dev/pandas/compare/018414e704670d2d727b9624089caae6f7d12ff1...90779ce4033b134fd8a3c3d26677ffda4c71bb50?src=pr)
Thanks. I added changes based on your review. Only building with python=2.7 fails. I have little to no experience with Python 2x. I didn't caught the TypeError in Python 2:
so

- some linting issues, use ``git diff master | flake8 --diff`` to show locally
- setup a venv or conda env with py2.7 to step thru the issues there.

Generally python 3 does the right thing with exceptions, while py2 may not be re-raising exceptions correctly (so your catching may not work here).

So you may need to have a slightly different strategy in py2 and py3 for catching the invalid args.

If you get stuck, ping.
Currently I'm designing new regexes for both py27 and py3k. I'll look into the linting. 
@hzpc-joostk 

>Currently I'm designing new regexes for both py27 and py3k. I'll look into the linting.

what do you mean? you can prob do something very straightforward, you don't even need regexes, you are just trying to find out if the error message if related to the calling of the function (and not the dtype of the data).
@jreback earlier you reviewed a part of my code:
> test where the callable has different than 1 arg (should be error)

In `.core.strings._map` these string methods are mapped. On both `TypeError` and `AttributeError`, `NaN` is returned instead. Using regex, implemented in commit 27065a29ba883f79499e4978872611f23d606f7e, only `TypeError`s regarding wrong number of args are reraised. However, these regexes only worked for Python 3. With e15dcdf544f02de04573b942132186d8ea397676, these regexes are unified into one sophisticated (complex) regex and work for Python 2 (tested).

tl;dr the tests should pass, as this now raises an error.
@hzpc-joostk lgtm. ping on green.
~Now~ Done fixing linting issues: 

    C:\Users\joostk\Documents\git\pandas>git diff 6ecc43d83801bd0337b771390ea5f4063689f275 | flake8 --diff
    .\pandas/core/strings.py:313:75: W291 trailing whitespace
    .\pandas/core/strings.py:314:70: W291 trailing whitespace
    .\pandas/core/strings.py:332:65: W291 trailing whitespace
    .\pandas/core/strings.py:341:69: W291 trailing whitespace
    .\pandas/core/strings.py:342:71: W291 trailing whitespace
    .\pandas/tests/test_strings.py:438:1: W293 blank line contains whitespace
    .\pandas/tests/test_strings.py:439:1: D102 Missing docstring in public method
    .\pandas/tests/test_strings.py:440:9: E266 too many leading '#' for block comment
    .\pandas/tests/test_strings.py:468:1: D102 Missing docstring in public method
@jreback Thanks for your input and help. All changes made. Waiting for CI... Besides that: all green!
https://travis-ci.org/pandas-dev/pandas/jobs/194493716

@hzpc-joostk 

so you can use

```
if PY2:
    # match py2 excetion
else:
    # match PY3 exception
```

and as I said you can do a minimal matching, because sometimes these exception messages change in versions (e.g. 3.5 -> 3.6 for example, maybe not in this case, but the wording can change). You are just generically interested in a specific type of error message (and not its specific wording per se)
thanks @hzpc-joostk 

nicely done!
Thanks @jreback 

Awesome. My first (accepted) PR on GitHub. 🍾 Cheers