genered_predictions.txt produced by run_summarization script may be of incorrect length
Author: JohnGiorgi
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.22.0.dev0
- Platform: Linux-4.18.0-372.19.1.el8_6.x86_64-x86_64-with-glibc2.30
- Python version: 3.9.6
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help?

@sgugger, @patil-suraj

### Information

- [X] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

This is straightforward enough that you can eyeball it. In the following code snippet from `run_summarization.py`, if `predict_with_generate` is `True` the script will save a file `"generated_predictions.txt"` containing the models predictions to disk 

https://github.com/huggingface/transformers/blob/adbf3a40de3524dcdce556914e2cb974d81854e5/examples/pytorch/summarization/run_summarization.py#L696-L704

If the model generates `"\n"` characters in its predictions (that don't occur at the beginning or end of the string), then the number of lines in `"generated_predictions.txt"` will not match the true number of model predictions.

### Expected behavior

`"generated_predictions.txt"` should contain a number of lines equal to the number of examples in the test set that we have model predictions for. Otherwise, this can cause problems downstream. E.g. I was burned by this when I tried to use `"generated_predictions.txt"` to submit to a leaderboard with a blind test set, and the number of predictions (i.e. lines in `"generated_predictions"`) didn't match the expected number.

Some possible solutions

1. Remove all newlines from the string, e.g. `[" ".join(pred.strip().split()) for pred in preds]`. This works, but is a little destructive as it would strip all whitespace characters besides single spaces.
2. Save the model predictions instead as a `json` or `jsonlines` file. This also works, but would be a breaking change in the sense that if someone using this script is currently expecting a text file, they would have to update their code to parse a json file.

If it is agreed this is a 🐛, I would be happy to make a PR with either of these approaches (or another approach)!


Comments:
Example scripts are just that, examples :-) They are not production-ready apps. They won't do everything you might need for your specific use-cases, but they are also easy to tweak as we try to keep them simple and readable. That's why we keep the generated text save simple, but you shouldn't hesitate to change the example to use either option 1 or 2, depending on what is easiest for you.

Yes I totally agree, didn't mean to suggest that they should be production ready. This was just a case that could obviously burn someone as the `generated_predictions.txt` file would be produced with this mistake silently and would only be detected if you checked its length against the expected number of predictions.

I am happy to use solution 1 in my own code, but I figured I would document this and suggest a general fix for the script itself so that others aren't burned by it -- especially because a model generating the newline character is not specific to just my situation.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.