Repository: stable-diffusion-webui
Issue: [Bug]: Interrogate CLIP Fails In BATCH Mode Code not handling Unicode Prompt characters
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

In the  Batch Interogate CLIP when writing the prompt containing Unicode characters to the txt file the code breaks with this error. UnicodeEncodeError: 'charmap' codec can't encode character '\u016b' in position 129: character maps to <undefined> And the batch process is stopped.

Traceback (most recent call last):
  File "J:\ImageGenerator\system\python\lib\site-packages\gradio\routes.py", line 422, in run_predict
    output = await app.get_blocks().process_api(
  File "J:\ImageGenerator\system\python\lib\site-packages\gradio\blocks.py", line 1323, in process_api
    result = await self.call_function(
  File "J:\ImageGenerator\system\python\lib\site-packages\gradio\blocks.py", line 1051, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "J:\ImageGenerator\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "J:\ImageGenerator\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "J:\ImageGenerator\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "J:\ImageGenerator\webui\modules\ui.py", line 989, in <lambda>
    fn=lambda *args: process_interrogate(interrogate, *args),
  File "J:\ImageGenerator\webui\modules\ui.py", line 158, in process_interrogate
    print(interrogation_function(img), file=open(os.path.join(ii_output_dir, f"{left}.txt"), 'a'))
  File "encodings\cp1252.py", line 19, in encode

### Steps to reproduce the problem

1. Go to the img2img tab 
2. Select the batch subtab
3. In the Input directory provide the folder containing the images
4. Press the Interrogate CLIP Button
5. This only happens when the image causes CLIP to generate a prompt containing a Unicode character. I am adding an image that generates a Unicode character in a prompt in my system.
![image_27](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/7073681/741fdbc3-4f31-4168-8748-57b87b4aa8ad)


### What should have happened?

The Batch Interogate CLIP Should write the prompt with Unicode characters in the Txt file.  And continue to the next image.

### Version or Commit where the problem happens

version: v1.4.0-RC

### What Python version are you running on ?

Python 3.10.x

### What platforms do you use to access the UI ?

Windows

### What device are you running WebUI on?

Nvidia GPUs (GTX 16 below)

### Cross attention optimization

Automatic

### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
--xformers --precision full --no-half --medvram
```


### List of extensions

<html>
<body>
<!--StartFragment-->

multidiffusion-upscaler-for-automatic1111 | https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111.git | main | de488810 | Sun Jun 18 21:39:44 2023 | unknown
-- | -- | -- | -- | -- | --
sd-webui-agent-scheduler | https://github.com/ArtVentureX/sd-webui-agent-scheduler.git | main | 8198489f | Thu Jun 29 03:24:05 2023 | unknown
sd-webui-controlnet | https://github.com/Mikubill/sd-webui-controlnet.git | main | 2785cbe6 | Sun Jul 2 05:20:52 2023 | unknown
stable-diffusion-webui-images-browser | https://github.com/AlUlkesh/stable-diffusion-webui-images-browser.git | main | b2f6e4cb | Thu Jun 8 08:11:43 2023 | unknown
ultimate-upscale-for-automatic1111 | https://github.com/Coyote-A/ultimate-upscale-for-automatic1111.git | master | c99f382b | Tue Jun 13 04:29:35 2023 | unknown

<!--EndFragment-->
</body>
</html>multidiffusion-upscaler-for-automatic1111	https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111.git	main	[de488810](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111/commit/de4888103c03d88c0d32af6d453373b47291687a)	Sun Jun 18 21:39:44 2023	unknown
sd-webui-agent-scheduler	https://github.com/ArtVentureX/sd-webui-agent-scheduler.git	main	[8198489f](https://github.com/ArtVentureX/sd-webui-agent-scheduler/commit/8198489fd42883d3af119e219c320ba4853802c8)	Thu Jun 29 03:24:05 2023	unknown
sd-webui-controlnet	https://github.com/Mikubill/sd-webui-controlnet.git	main	[2785cbe6](https://github.com/Mikubill/sd-webui-controlnet/commit/2785cbe61a61c137d4e980752771f6329b03612a)	Sun Jul 2 05:20:52 2023	unknown
stable-diffusion-webui-images-browser	https://github.com/AlUlkesh/stable-diffusion-webui-images-browser.git	main	[b2f6e4cb](https://github.com/AlUlkesh/stable-diffusion-webui-images-browser/commit/b2f6e4cbe507078e3bf2b9d5fce6f83e2e233aa8)	Thu Jun 8 08:11:43 2023	unknown
ultimate-upscale-for-automatic1111	https://github.com/Coyote-A/ultimate-upscale-for-automatic1111.git	master	[c99f382b](https://github.com/Coyote-A/ultimate-upscale-for-automatic1111/commit/c99f382b31509b87b4d512e70e9caf08ae7a079f)	Tue Jun 13 04:29:35 2023	unknown

### Console logs

```Shell
load checkpoint from J:\ImageGenerator\webui\models\BLIP\model_base_caption_capfilt_large.pth
Traceback (most recent call last):
  File "J:\ImageGenerator\system\python\lib\site-packages\gradio\routes.py", line 422, in run_predict
    output = await app.get_blocks().process_api(
  File "J:\ImageGenerator\system\python\lib\site-packages\gradio\blocks.py", line 1323, in process_api
    result = await self.call_function(
  File "J:\ImageGenerator\system\python\lib\site-packages\gradio\blocks.py", line 1051, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "J:\ImageGenerator\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "J:\ImageGenerator\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "J:\ImageGenerator\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "J:\ImageGenerator\webui\modules\ui.py", line 989, in <lambda>
    fn=lambda *args: process_interrogate(interrogate, *args),
  File "J:\ImageGenerator\webui\modules\ui.py", line 158, in process_interrogate
    print(interrogation_function(img), file=open(os.path.join(ii_output_dir, f"{left}.txt"), 'a'))
  File "encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\u016b' in position 129: character maps to <undefined>
```


### Additional information

This only happens when the image causes CLIP to generate a prompt containing a Unicode character. I am adding an image that generates a Unicode character in a prompt in my system.