BUG: to_json - prevent various segfault conditions (GH14256)
State: closed

- [x] closes #14256 
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

There were several sources for the JSON string buffer at enc->start exceeding the reserved space:
- Loops on DataFrame columns being stuck on the same column if enc->errorMsg was set. This is fixed by adding the check for the errorMsg to the respective iterNext function.
- Loops on objects (Dir_iterNext) not breaking due to similar reasons, which allows infinite recursion. Also added check for errorMsg.
- Column labels were added in iterName methods inside objToJSON.c where the check for remaining buffer was not accessible. Moved Buffer_Reserve to the ujson header file.
- Closing brackets could be added without sufficient buffer. Added Buffer_Reserve calls.


Labels:
name: Bug
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=h1) Report
> Merging [#17857](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c277cd76416d4e930b1f05da873b9eaf101139da?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17857/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17857      +/-   ##
==========================================
- Coverage   91.22%    91.2%   -0.02%     
==========================================
  Files         163      163              
  Lines       50069    50038      -31     
==========================================
- Hits        45673    45639      -34     
- Misses       4396     4399       +3
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.01% <ø> (ø)` | :arrow_up: |
| #single | `40.26% <ø> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/dtypes/concat.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2NvbmNhdC5weQ==) | `98.26% <0%> (-0.87%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.75% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.48% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.38% <0%> (-0.07%)` | :arrow_down: |
| [pandas/core/sparse/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL3Nlcmllcy5weQ==) | `95.26% <0%> (-0.02%)` | :arrow_down: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `92.2% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/dtypes/dtypes.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2R0eXBlcy5weQ==) | `95.14% <0%> (ø)` | :arrow_up: |
| [pandas/core/reshape/concat.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9jb25jYXQucHk=) | `97.6% <0%> (+0.03%)` | :arrow_up: |
| [pandas/core/indexing.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhpbmcucHk=) | `93% <0%> (+0.18%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=footer). Last update [c277cd7...41a1aa0](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=h1) Report
> Merging [#17857](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/3c964a47d626a06a3f9c2d0795ee7d744dc72363?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17857/graphs/tree.svg?token=eZ4WkYLtcO&height=150&src=pr&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17857      +/-   ##
==========================================
+ Coverage   91.23%   91.24%   +<.01%     
==========================================
  Files         163      163              
  Lines       50075    50075              
==========================================
+ Hits        45688    45691       +3     
+ Misses       4387     4384       -3
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.05% <ø> (+0.02%)` | :arrow_up: |
| #single | `40.29% <ø> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.75% <0%> (-0.1%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.2% <0%> (+1.81%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=footer). Last update [3c964a4...747a942](https://codecov.io/gh/pandas-dev/pandas/pull/17857?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Thank you for the comments. I moved the test comments and the whatsnew entry, and pushed the changes.
@matthiashuschle also pls rebase on master. some CI things were updated to make circleci work with the new version of mpl.
thanks, I just incorporated your suggestions.
thanks @matthiashuschle nice patch!