Repository: optuna/optuna

Issue:
`RDBStorage` does not sort trails with postgres
### Expected behavior

`RDBStorage.get_all_trials()` should sort trials like `CachedStorage` do when using postgres.
Also, it would be great if `intermediate_values` were sorted too.

### Environment

- Optuna version: 2.10.0
- Python version: 3.8.13
- OS: Linux-3.10.0-1160.62.1.el7.x86_64-x86_64-with-glibc2.2.5

### Error messages, stack traces, or logs

```shell
See steps to reproduce.
```


### Steps to reproduce

1. Perform a study with `postgresql+psycopg2://...` storage.
2. Get trails from `CachedStorage` and `RDBStorage` by your study id.

```python
>>> import optuna

>>> storage = optuna.storages.get_storage('postgresql+psycopg2://...')
>>> [t.number for t in storage.get_all_trials(study_id=...)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, ...]
>>> storage.__class__
<class 'optuna.storages._cached_storage._CachedStorage'>

>>> storage = optuna.storages.RDBStorage('postgresql+psycopg2://...')
>>> [t.number for t in storage.get_all_trials(study_id=...)]
[33, 34, 28, 38, 5, 36, 1, 11, 10, 9, 8, 7, 14, 24, 42, 32, 17, 40, 43, 41, 49, 25, 54, 58, 48, ...]
>>> storage.__class__
<class 'optuna.storages._rdb.storage.RDBStorage'>
```


### Additional context (optional)

Found this while fixing plots https://github.com/optuna/optuna-dashboard/pull/235

https://www.postgresql.org/docs/current/sql-select.html says:
> If `ORDER BY` is not given, the rows are returned in whatever order the system finds fastest to produce.


Discussion:
@pnkov Thank you for reporting a bug! I executed a following script with PostgreSQL 14.3 server and Optuna 2.10.0.

```python
import optuna


def objective(trial):
    x = trial.suggest_float("x", -100, 100)
    y = trial.suggest_categorical("y", [-1, 0, 1])
    return x**2 + y


def main():
    storage = optuna.storages.RDBStorage("postgresql+psycopg2://root:root@localhost:5432/optuna")
    study = optuna.create_study(storage=storage)
    study.optimize(objective, n_trials=20)

    print("Print the order of trial's number")
    print([t.number for t in storage.get_all_trials(study_id=study._study_id)])


if __name__ == '__main__':
    main()
```

Then I got a following output.

```
$ python example.py
[I 2022-06-08 10:00:04,328] A new study created in RDB with name: no-name-fe132448-7fdf-4c38-9a59-aa5611fc252e
[I 2022-06-08 10:00:04,545] Trial 0 finished with value: 6509.35468355771 and parameters: {'x': 80.68676894979566, 'y': -1}. Best is trial 0 with value: 6509.35468355771.
[I 2022-06-08 10:00:04,678] Trial 1 finished with value: 2255.1446289317128 and parameters: {'x': -47.49889081790977, 'y': -1}. Best is trial 1 with value: 2255.1446289317128.
[I 2022-06-08 10:00:04,786] Trial 2 finished with value: 6215.957053977707 and parameters: {'x': 78.84768261640735, 'y': -1}. Best is trial 1 with value: 2255.1446289317128.
[I 2022-06-08 10:00:04,908] Trial 3 finished with value: -0.4226857153793956 and parameters: {'x': 0.7598120061045393, 'y': -1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,031] Trial 4 finished with value: 3037.5199827529195 and parameters: {'x': -55.10462759835075, 'y': 1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,157] Trial 5 finished with value: 6567.853338849351 and parameters: {'x': -81.03612366623511, 'y': 1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,286] Trial 6 finished with value: 2224.548011208507 and parameters: {'x': 47.15451209808566, 'y': 1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,404] Trial 7 finished with value: 7954.292436683009 and parameters: {'x': 89.19244607410994, 'y': -1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,529] Trial 8 finished with value: 20.05114235824884 and parameters: {'x': 4.364761431997039, 'y': 1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,675] Trial 9 finished with value: 4878.94958438663 and parameters: {'x': -69.85663593665694, 'y': -1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,803] Trial 10 finished with value: 8.99239351941829 and parameters: {'x': -2.9987319852594845, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:05,927] Trial 11 finished with value: 44.348844323820686 and parameters: {'x': -6.659492797790286, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,057] Trial 12 finished with value: 2.616557950356071 and parameters: {'x': 1.6175778034938755, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,179] Trial 13 finished with value: 849.360507860521 and parameters: {'x': 29.143790210961253, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,329] Trial 14 finished with value: 1115.8944361446668 and parameters: {'x': -33.405006153938466, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,447] Trial 15 finished with value: 1068.5556749114485 and parameters: {'x': 32.68876985925669, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,565] Trial 16 finished with value: 739.4461417687749 and parameters: {'x': -27.192758995158524, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,702] Trial 17 finished with value: 285.0587446986398 and parameters: {'x': 16.913271259535804, 'y': -1}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,825] Trial 18 finished with value: 3176.3341620785336 and parameters: {'x': 56.35897587854603, 'y': 0}. Best is trial 3 with value: -0.4226857153793956.
[I 2022-06-08 10:00:06,938] Trial 19 finished with value: 9487.144930451354 and parameters: {'x': -97.40710923978472, 'y': -1}. Best is trial 3 with value: -0.4226857153793956.
Print the order of trial's number
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
```

So I cannot reproduce the bug. You can reproduce my experiment in following steps:

```
$ git clone git@github.com:c-bata/optuna-issue-3605.git
$ cd optuna-issue-3605
$ python -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt
$ ./test_postgresql_order.sh
```

https://github.com/c-bata/optuna-issue-3605

Could you tell me more detailed condition and/or requirement to reproduce the bug?
@c-bata Thank you for providing an example. I've managed to modify it to force the bug appearance by concurrency and requesting immediate vacuum from postgres. In theory, none of that is necessary due to postgres docs. In practice, the main reasons of changed output order are `update` and `delete` statements and scheduled `autovacuum` executions.

```python
import optuna


def objective(trial):
    x = trial.suggest_float("x", -100, 100)
    y = trial.suggest_categorical("y", [-1, 0, 1])
    time.sleep(0.20 - trial.number / 100)
    return x**2 + y


def main():
    storage = optuna.storages.RDBStorage("postgresql+psycopg2://root:root@localhost:5432/optuna")
    study = optuna.create_study(storage=storage)
    study.optimize(objective, n_trials=20, n_jobs=4)

    with storage.engine.connect() as connection:
        connection.execute("COMMIT")
        connection.execute("VACUUM ANALYZE trials")

    print("Print the order of trial's number")
    print([t.number for t in storage.get_all_trials(study_id=study._study_id)])


if __name__ == '__main__':
    main()
```

And my output looks like this:

```
[I 2022-06-14 22:05:39,767] A new study created in RDB with name: no-name-b10e827d-1fa7-426d-8338-96a434ef4ba2
/usr/local/lib/python3.9/site-packages/optuna/study/study.py:393: FutureWarning: `n_jobs` argument has been deprecated in v2.7.0. This feature will be removed in v4.0.0. See https://github.com/optuna/optuna/releases/tag/v2.7.0.
  warnings.warn(
[I 2022-06-14 22:05:40,279] Trial 2 finished with value: 9880.242842154103 and parameters: {'x': 99.40444075670916, 'y': -1}. Best is trial 2 with value: 3782.2022818644314.
[I 2022-06-14 22:05:40,314] Trial 3 finished with value: 6411.302422839392 and parameters: {'x': -80.06436425051655, 'y': 1}. Best is trial 1 with value: 3782.2022818644314.
[I 2022-06-14 22:05:40,335] Trial 0 finished with value: 4995.306898451873 and parameters: {'x': -70.67041034585743, 'y': 1}. Best is trial 1 with value: 3782.2022818644314.
[I 2022-06-14 22:05:40,342] Trial 1 finished with value: 3782.2022818644314 and parameters: {'x': 61.491481376402305, 'y': 1}. Best is trial 1 with value: 3782.2022818644314.
[I 2022-06-14 22:05:40,719] Trial 4 finished with value: 4505.268996787651 and parameters: {'x': -67.11385100549403, 'y': 1}. Best is trial 1 with value: 2107.015948215241.
[I 2022-06-14 22:05:40,731] Trial 7 finished with value: 5765.795193419933 and parameters: {'x': 75.92624838236071, 'y': 1}. Best is trial 1 with value: 2107.015948215241.
[I 2022-06-14 22:05:40,746] Trial 6 finished with value: 2107.015948215241 and parameters: {'x': 45.89134938324696, 'y': 1}. Best is trial 6 with value: 2107.015948215241.
[I 2022-06-14 22:05:40,759] Trial 5 finished with value: 9885.093164093316 and parameters: {'x': -99.42883467130305, 'y': -1}. Best is trial 6 with value: 2107.015948215241.
[I 2022-06-14 22:05:41,017] Trial 11 finished with value: 1814.427926822495 and parameters: {'x': -42.5961022491788, 'y': 0}. Best is trial 11 with value: 1814.427926822495.
[I 2022-06-14 22:05:41,047] Trial 8 finished with value: 6782.281790087273 and parameters: {'x': 82.36068109290545, 'y': -1}. Best is trial 11 with value: 1749.5151887696472.
[I 2022-06-14 22:05:41,076] Trial 10 finished with value: 1749.5151887696472 and parameters: {'x': 41.81525067208909, 'y': 1}. Best is trial 10 with value: 1749.5151887696472.
[I 2022-06-14 22:05:41,085] Trial 9 finished with value: 9181.234968563425 and parameters: {'x': -95.81354272003215, 'y': 1}. Best is trial 10 with value: 1749.5151887696472.
[I 2022-06-14 22:05:41,225] Trial 12 finished with value: 213.70099399732393 and parameters: {'x': -14.618515451211998, 'y': 0}. Best is trial 12 with value: 213.70099399732393.
[I 2022-06-14 22:05:41,291] Trial 14 finished with value: 81.93838282022969 and parameters: {'x': -9.051982259164546, 'y': 0}. Best is trial 14 with value: 81.93838282022969.
[I 2022-06-14 22:05:41,335] Trial 15 finished with value: 403.37159344341103 and parameters: {'x': -20.08411296132869, 'y': 0}. Best is trial 14 with value: 81.93838282022969.
[I 2022-06-14 22:05:41,344] Trial 13 finished with value: 283.500599106398 and parameters: {'x': -16.837476031353333, 'y': 0}. Best is trial 14 with value: 81.93838282022969.
[I 2022-06-14 22:05:41,430] Trial 16 finished with value: 29.550342862761102 and parameters: {'x': -5.436022706240391, 'y': 0}. Best is trial 16 with value: 29.550342862761102.
[I 2022-06-14 22:05:41,500] Trial 17 finished with value: 212.7718005133823 and parameters: {'x': 14.586699438645546, 'y': 0}. Best is trial 16 with value: 29.550342862761102.
[I 2022-06-14 22:05:41,503] Trial 19 finished with value: 257.5906222779072 and parameters: {'x': 16.049629973239483, 'y': 0}. Best is trial 16 with value: 29.550342862761102.
[I 2022-06-14 22:05:41,512] Trial 18 finished with value: 46.82363010740482 and parameters: {'x': 6.842779413908125, 'y': 0}. Best is trial 16 with value: 29.550342862761102.
Print the order of trial's number
[2, 0, 3, 1, 4, 7, 6, 5, 11, 8, 9, 10, 12, 14, 15, 13, 16, 17, 19, 18]

```
@pnkov Thank you for your reply! I confirm the bug is able to be reproduced in my environment.

https://github.com/c-bata/optuna-issue-3605/blob/main/example.py

```
$ python example.py
[I 2022-06-15 09:54:27,738] A new study created in RDB with name: no-name-ca7f3e66-b32b-40a7-a6b2-acfcb5c18852
/Users/c-bata/sandbox/optuna-issue-3605/venv/lib/python3.8/site-packages/optuna/study/study.py:393: FutureWarning: `n_jobs` argument has been deprecated in v2.7.0. This feature will be removed in v4.0.0. See https://github.com/optuna/optuna/releases/tag/v2.7.0.
  warnings.warn(
[I 2022-06-15 09:54:28,367] Trial 1 finished with value: 1578.06320530163 and parameters: {'x': 39.72484367875637, 'y': 0}. Best is trial 1 with value: 262.5038998124118.
[I 2022-06-15 09:54:28,372] Trial 2 finished with value: 262.5038998124118 and parameters: {'x': -16.171082209067265, 'y': 1}. Best is trial 2 with value: 262.5038998124118.
[I 2022-06-15 09:54:28,519] Trial 3 finished with value: 1992.2566836524625 and parameters: {'x': 44.623499231374296, 'y': 1}. Best is trial 2 with value: 262.5038998124118.
[I 2022-06-15 09:54:28,566] Trial 0 finished with value: 2126.12171141445 and parameters: {'x': -46.10988734983474, 'y': 0}. Best is trial 2 with value: 262.5038998124118.
[I 2022-06-15 09:54:28,814] Trial 4 finished with value: 6057.849101607924 and parameters: {'x': 77.83860932472987, 'y': -1}. Best is trial 2 with value: 262.5038998124118.
[I 2022-06-15 09:54:28,858] Trial 5 finished with value: 1.6568217872925186 and parameters: {'x': -1.287175896019079, 'y': 0}. Best is trial 5 with value: 1.6568217872925186.
[I 2022-06-15 09:54:28,944] Trial 6 finished with value: 1.3489445408338332 and parameters: {'x': -0.5907152789913539, 'y': 1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:28,964] Trial 7 finished with value: 6628.511691584193 and parameters: {'x': 81.41567227250655, 'y': 0}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,127] Trial 8 finished with value: 4110.001179403038 and parameters: {'x': 64.11708960490205, 'y': -1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,134] Trial 9 finished with value: 3683.83852763852 and parameters: {'x': 60.68639491383979, 'y': 1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,230] Trial 10 finished with value: 1931.8928782228495 and parameters: {'x': 43.96467761991266, 'y': -1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,259] Trial 11 finished with value: 3735.4750331707687 and parameters: {'x': 61.11035127677445, 'y': 1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,422] Trial 13 finished with value: 7730.576028581793 and parameters: {'x': -87.92369435244287, 'y': 0}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,423] Trial 12 finished with value: 9201.656796781575 and parameters: {'x': -95.92526672770619, 'y': 0}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,450] Trial 14 finished with value: 5.00343550861376 and parameters: {'x': 2.236836048666455, 'y': 0}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,475] Trial 15 finished with value: 8704.037105399202 and parameters: {'x': -93.29542917742113, 'y': 0}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,718] Trial 16 finished with value: 30.413893622343526 and parameters: {'x': 5.423457718314353, 'y': 1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,759] Trial 17 finished with value: 1.6691237128484278 and parameters: {'x': -0.8179998244794603, 'y': 1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,760] Trial 18 finished with value: 98.77592922804851 and parameters: {'x': 9.888171177121102, 'y': 1}. Best is trial 6 with value: 1.3489445408338332.
[I 2022-06-15 09:54:29,760] Trial 19 finished with value: 67.60721877587436 and parameters: {'x': 8.161324572388624, 'y': 1}. Best is trial 6 with value: 1.3489445408338332.
Print the order of trial's number
[1, 2, 3, 0, 4, 5, 6, 7, 8, 9, 10, 11, 13, 12, 14, 15, 16, 18, 17, 19]
```

I thought PostgreSQL returns the records in the order of primary keys as InnoDB (MySQL) does. However, it is seemingly not ensured. I found a stackoverflow's page that mentioning about it.

> PostgreSQL tables have no defined order, with or without a primary key; they're a "heap" of rows arranged in page blocks. Ordering is imposed using the ORDER BY clause of queries when desired.
> https://stackoverflow.com/questions/13190720/what-is-the-order-of-records-in-a-table-with-a-composite-primary-key

Again, thank you for reporting this bug. I'll open a pull request to fix this bug.

Resolving patch:
diff --git a/optuna/storages/_rdb/storage.py b/optuna/storages/_rdb/storage.py
--- a/optuna/storages/_rdb/storage.py
+++ b/optuna/storages/_rdb/storage.py
@@ -997,6 +997,7 @@ def _get_trials(
                         models.TrialModel.trial_id.in_(trial_ids),
                         models.TrialModel.study_id == study_id,
                     )
+                    .order_by(models.TrialModel.trial_id)
                     .all()
                 )
             except sqlalchemy_exc.OperationalError as e:
@@ -1018,6 +1019,7 @@ def _get_trials(
                     .options(sqlalchemy_orm.selectinload(models.TrialModel.system_attributes))
                     .options(sqlalchemy_orm.selectinload(models.TrialModel.intermediate_values))
                     .filter(models.TrialModel.study_id == study_id)
+                    .order_by(models.TrialModel.trial_id)
                     .all()
                 )
                 trial_models = [t for t in trial_models if t.trial_id in trial_ids]


Test patch: