WebSearchTool citation markers incorrectly positioned for multibyte text

### What happened?

The WebSearchTool inserts citation markers (e.g., `[1]`, `[2]`) at incorrect positions when the search response contains multibyte characters such as Japanese text.
The Grounding API returns byte positions for the segments, but the current implementation treats these as character positions, causing markers to appear in the middle of text instead of at the intended locations.

This bug can be reproduced by running a web search query that returns multibyte characters.
https://gist.github.com/pokutuna/e07fe0bbb5c26b419f4fcd89c8800371

---

```
QUERY: Can you suggest Japanese proverbs that effectively express bugs caused by language differences and encoding issues?'

TOOL_RESPONSE:
Web search results for "Can you suggest Japanese proverbs that effectively express bugs caused by language differences and encoding issues?":

Of course. Here are several Japanese proverbs and concepts that can effectively and creatively describe bugs caused by language differences and encoding issues.

...

Here are some proverbs that can be used metaphorically:

**1. 井の中の蛙大海を知らず (I no naka no kawazu taikai o shirazu)**
*   **Literal Translation:** [1][2]"A frog in a well knows nothing of the great ocean."
*   **How it applies:** This powerfully describes the root cause of many internationalization bugs. A developer who has only ever worked with a single language and character set is the "frog in the well." They are unaware of the "great ocean" of different languages, scripts, and encodings, and therefore write code that fails when it encounters them.

**2. 二階から目薬 (Nikai kara megusuri)**
*   **Literal Translation:** "Eyedrops fr[3]om the second floor."
*   **How it applies:** This proverb describes a method that is too roundabout and ultimately ineffective. It's a perfect metaphor for a clumsy attempt to fix an encoding bug without understanding the root cause—for instance, trying to manually replace garbled characters instead of correctly setting the encoding headers. The effort is futile because the approach is fundamentally wrong.

**3. ミイラ取りがミイラになる (Miira tori ga miira ni naru)**
*   **Literal Translation:** "The mummy hunter becomes a mummy."
* [3]  **How it applies:** This humorously describes a developer who dives deep into a complex encoding or internationalization bug. They get so entangled in debugging corrupted data and obscure character sets that their own code becomes a convoluted, unmaintainable mess, effectively "becoming a mummy" themselves.

Sources:
[1] fluentin3months.com (https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGQYPuKiAiJKRzkaWm-5zEr7SCemUtSzR9bQXw1Z8Yba6vF4meuAIOWK-ec2FS7XIf1TGsHQRALEzV_J-kr1yzr5kVUdllS8z6yEtpGSSL5jL_qLV9nw-sOZSWUcsNgnh20DQUu2MiEGatsN9FO)
[2] tandem.net (https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGf6j2itteHi0CdHoWKkMUlCOSctlDEHwaAABTzjb77eDPQsmiYCTnDwhdcfzq-S2gH1eUujJLRrOg5vUHr352ExZcXpsR7eS-rrnlddvJsYO5g_wIfOAZZHz6dZJU9KWBgYB_r)
[3] nihonshock.com (https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFFYpv0TbaxKP-WXPqb69gQ917bzS0oMyFTRor397JRi8EKnh0qOy17iHWtbUlJNqiTyBjQ9hpkfIER5-5KSrpi2clPfS22WAIPB0VuLso3i04TjDnplKFHllIZWAj2cSGhIN6bRQj2jX4_Vi0NJIATPg2R23jtwmn7)
```


### What did you expect to happen?

Citation markers should be inserted at the correct positions as intended by the API response, appearing at natural text boundaries (e.g., word endings, sentence endings) rather than mid-word or within formatting.

### Client information

<details>

```console
$ gemini /about
╭─────────────────────────────────────────────────────────────────────╮
│                                                                     │
│ About Gemini CLI                                                    │
│                                                                     │
│ CLI Version             0.1.18                                      │
│ Git Commit              cd7e60e0                                    │
│ Model                   gemini-2.5-pro                              │
│ Sandbox                 no sandbox                                  │
│ OS                      darwin                                      │
│ Auth Method             vertex-ai                                   │
│ GCP Project             pokutuna-playground                           │
│                                                                     │
╰─────────────────────────────────────────────────────────────────────╯
```

</details>


### Login information

Reproduces with both Vertex AI and API key authentication methods.

### Anything else we need to know?

While users may rarely see malformed output since Gemini likely removes unnatural citations from tool responses, avoiding corrupted input to the LLM is still beneficial for improving output quality and citation preservation.

In `packages/core/src/tools/web-search.ts`,  the code uses `modifiedResponseText.split('')` to create a character array and then uses `splice()` with byte indices from the API response. This causes incorrect positioning when multibyte UTF-8 characters are present, as JavaScript's `split('')` operates on Unicode code points while the API provides byte offsets.


These documents show that the startIndex and endIndex of the segment are byte positions.
- [Gemini API Segment](https://ai.google.dev/api/generate-content#Segment)
- [Vertex AI GroundingSupport](https://cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1beta1/GroundingMetadata#GroundingSupport)

Although the encoding of the target text is not explicitly specified, it would be reasonable to consider it as UTF-8. When I made modifications to handle it as UTF-8, the marker positions became natural.

Labels: priority/p2
State: closed