Repository: sphinx-doc/sphinx

Issue:
searchindex.js: parallel HTML builds may produce non-deterministic index
### Describe the bug

This bug is a follow-up / companion bug to #11622.

In cases where the HTML builder runs with [parallel writes allowed](https://github.com/sphinx-doc/sphinx/blob/882a174e48a4dfd22d4fab4b2e3b74f091b3f98e/sphinx/builders/__init__.py#L349-L353), then the timing of calls to `index_page` (that in turn calls [`IndexBuilder.feed`](https://github.com/sphinx-doc/sphinx/blob/882a174e48a4dfd22d4fab4b2e3b74f091b3f98e/sphinx/builders/html/__init__.py#L942-L945) and importantly populates [`IndexBuilder._alltitles`](https://github.com/sphinx-doc/sphinx/blob/882a174e48a4dfd22d4fab4b2e3b74f091b3f98e/sphinx/search/__init__.py#L445)) is unpredictable.

For cases where `search` is enabled (the default), the nondeterministic ordering of the `IndexBuilder._alltitles` dictionary is then reflected in the serialized `searchindex.js` file contents.

For reproducible results, this is undesirable behaviour.

### How to Reproduce

On a system with more than one processor:

```sh
$ git clone https://github.com/quodlibet/quodlibet/ && cd quodlibet
$ python3 -m venv .venv && source .venv/bin/activate
$ pip install sphinx==7.2.6 sphinx-rtd-theme==2.0.0
$ cd docs
$ for _ in $(seq 10); do rm -rf _build_all; make > /dev/null; sha256sum _build_all/searchindex.js; done;
... # observe varying hash outputs
```

One prerequisite for this bug to occur that `quodlibet` exhibits is that the `Makefile` [auto-configures the `jobs` command-line flag](https://github.com/quodlibet/quodlibet/blob/0ecc4a23e7ca92d5e3c12be127ec64a2f21342e2/docs/Makefile#L1-L2) to enable parallelism when available.

### Environment Information

```text
Platform:              linux; (Linux-6.5.0-5-amd64-x86_64-with-glibc2.37)
Python version:        3.11.7 (main, Dec  8 2023, 14:22:46) [GCC 13.2.0])
Python implementation: CPython
Sphinx version:        7.2.6
Docutils version:      0.20.1
Jinja2 version:        3.1.3
Pygments version:      2.17.2
```


### Sphinx extensions

```python
["sphinx.ext.autodoc", "sphinx.ext.extlinks", "contributors"]
```


### Additional context

_No response_

Discussion:

Patch:
diff --git a/sphinx/search/__init__.py b/sphinx/search/__init__.py
index 99c934a0f..42b45e113 100644
--- a/sphinx/search/__init__.py
+++ b/sphinx/search/__init__.py
@@ -392,7 +392,7 @@ def freeze(self) -> dict[str, Any]:
         objnames = self._objnames
 
         alltitles: dict[str, list[tuple[int, str]]] = {}
-        for docname, titlelist in self._all_titles.items():
+        for docname, titlelist in sorted(self._all_titles.items()):
             for title, titleid in titlelist:
                 alltitles.setdefault(title, []).append((fn2index[docname], titleid))
 


Test patch:
diff --git a/tests/test_search.py b/tests/test_search.py
index d4bbef5b3..f13c9a246 100644
--- a/tests/test_search.py
+++ b/tests/test_search.py
@@ -157,8 +157,8 @@ def test_IndexBuilder():
     index = IndexBuilder(env, 'en', {}, None)
     index.feed('docname1_1', 'filename1_1', 'title1_1', doc)
     index.feed('docname1_2', 'filename1_2', 'title1_2', doc)
-    index.feed('docname2_1', 'filename2_1', 'title2_1', doc)
     index.feed('docname2_2', 'filename2_2', 'title2_2', doc)
+    index.feed('docname2_1', 'filename2_1', 'title2_1', doc)
     assert index._titles == {'docname1_1': 'title1_1', 'docname1_2': 'title1_2',
                              'docname2_1': 'title2_1', 'docname2_2': 'title2_2'}
     assert index._filenames == {'docname1_1': 'filename1_1', 'docname1_2': 'filename1_2',