Repository: pandas-dev/pandas
Pull request: BUG: Fix json_normalize throwing TypeError (#21536)
Fix json_normalize throwing TypeError with array of values and record_prefix (#21536)

- [x] closes #21536
- [x] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
json_normalize throws `TypeError` with array of values and `record_prefix`
#### Code Sample, a copy-pastable example if possible

```python
from pandas.io.json import json_normalize

df = json_normalize({'A': [1, 2]}, 'A', record_prefix='Prefix.')
print(df)

```
#### Problem description

The above code throws a `TypeError`:
```
Traceback (most recent call last):
  File "c:\Users\levu\Desktop\tmp\json_normalize\main.py", line 3, in <module>
    df = json_normalize({'A': [1, 2]}, 'A', record_prefix='Prefix.')
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 262, in json_normalize
    result.rename(columns=lambda x: record_prefix + x, inplace=True)
  File "C:\Python36\lib\site-packages\pandas\util\_decorators.py", line 187, in wrapper
    return func(*args, **kwargs)
  File "C:\Python36\lib\site-packages\pandas\core\frame.py", line 3781, in rename
    return super(DataFrame, self).rename(**kwargs)
  File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 973, in rename
    level=level)
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 3340, in rename_axis
    obj.set_axis(axis, _transform_index(self.axes[axis], mapper, level))
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 5298, in _transform_index
    items = [func(x) for x in index]
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 5298, in <listcomp>
    items = [func(x) for x in index]
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 262, in <lambda>
    result.rename(columns=lambda x: record_prefix + x, inplace=True)
TypeError: must be str, not int
```

I think line 262 in `normalize.py` should be:
```
result.rename(columns=lambda x: "{p}{c}".format(p=record_prefix,c=x), inplace=True)
```
because `x` can be integer.

#### Expected Output
| |Prefix.0|
|-|-|
|0|1|
|1|2|
#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 62 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.1
pytest: 3.6.1
pip: 10.0.1
setuptools: 28.8.0
Cython: None
numpy: 1.14.2
scipy: None
pyarrow: None
xarray: None
IPython: 6.3.1
sphinx: None
patsy: None
dateutil: 2.7.2
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

</details>

Hints from issue discussion:
That indeed looks suspicious!  PR to patch is welcome!

cc @WillAyd 
@vuminhle : Marking this for `0.23.2`, as @vuminhle has already identified a potential fix, which we can easily check and patch if this actually works if no one picks this up.

Commit message: BUG: Fix json_normalize throws TypeError with array of values and ecord_prefix (#21536)

Patch to review:
diff --git a/pandas/io/json/normalize.py b/pandas/io/json/normalize.py
index b845a43b9ca9e..6ca189fe9c77e 100644
--- a/pandas/io/json/normalize.py
+++ b/pandas/io/json/normalize.py
@@ -259,7 +259,8 @@ def _recursive_extract(data, path, seen_meta, level=0):
     result = DataFrame(records)
 
     if record_prefix is not None:
-        result.rename(columns=lambda x: record_prefix + x, inplace=True)
+        result.rename(columns=lambda x: "{p}{c}".format(p=record_prefix, c=x),
+                      inplace=True)
 
     # Data types, a problem
     for k, v in compat.iteritems(meta_vals):
diff --git a/pandas/tests/io/json/test_normalize.py b/pandas/tests/io/json/test_normalize.py
index 395c2c90767d3..5f9a356af6daf 100644
--- a/pandas/tests/io/json/test_normalize.py
+++ b/pandas/tests/io/json/test_normalize.py
@@ -123,6 +123,12 @@ def test_simple_normalize_with_separator(self, deep_nested):
                           'country', 'states_name']).sort_values()
         assert result.columns.sort_values().equals(expected)
 
+    def test_value_array_record_prefix(self):
+        # GH 21536
+        result = json_normalize({'A': [1, 2]}, 'A', record_prefix='Prefix.')
+        expected = DataFrame([[1], [2]], columns=['Prefix.0'])
+        tm.assert_frame_equal(result.reindex_like(expected), expected)
+
     def test_more_deeply_nested(self, deep_nested):
 
         result = json_normalize(deep_nested, ['states', 'cities'],


Code review comments:
diff_hunk: @@ -259,7 +259,8 @@ def _recursive_extract(data, path, seen_meta, level=0):
     result = DataFrame(records)
 
     if record_prefix is not None:
-        result.rename(columns=lambda x: record_prefix + x, inplace=True)
+        result.rename(columns=lambda x: "{p}{c}".format(p=record_prefix, c=x),
path: pandas/io/json/normalize.py
text: @user1:
can you change to 

``result = result.rename(...)``

we don't use ``inplace`` in library code