Repository: pytorch/ignite

Issue:
"=" in the checkpoint name may be incompatible with OSs
## ❓ Questions/Help/Support
Issue raised by Eric from MONAI team:
File names with operators in them may be valid names in Linux but that's not the case in other OSs, plus shells like bash really don't like them.

Discussion:
vfdev-5: @Joel-hanson thanks for the PR ! Let me check it and comment out things I do not understand...
Joel-hanson: Looking into the test which are failed

Joel-hanson: > Looking into the test which failed.

Fixed some of the tests which were related to the current implementation

@vfdev-5 @sdesrozis can you please help me fix the other test cases
vfdev-5: @Joel-hanson our CI can fail on distributed tests, so no worries for failed tests...
 I restarted failed test on Travis 
vfdev-5: @Joel-hanson thanks for working on it. I left several comments that may improve current implementation and open a small discussion about improvements of `Checkpoint` class itself. 
vfdev-5: @Joel-hanson the problem of using predefined pattern like `{filename_prefix}_{name}_{global_step}_{score_name}={score}.{ext}` forces additions and suppressions because actual default pattern is more like `{filename_prefix}{name}_{global_step}{score_name}{score}.{ext}` with special processing of fields.

Just tried to play around all of that, I think class attribute is not a good approach... For example, I would like to setup 2 savers: training checkpoint (to save trainer, model, optimizer etc) and best model. For the first one, I do not want to alter filename pattern and for the second I provide score name and score function but for the filename instead of `best_model_9_val_acc=0.77.pt` I would like to have `best_model_9.pt`...

Maybe, we can introduce it as an additional argument to `__init__` with default as `None`.

Joel-hanson: Good catch @vfdev-5 

@vfdev-5 @sdesrozis 
Just to be clear for me I will note down what I have in mind for the new implementation introducing additional argument to `__init__`.

- I will remove the class attributes and will introduce a new additional argument called `filename_pattern`
- Will also make user-defined attributes as public
```
self._fname_prefix -> self.filename_prefix
self._score_function -> self.score_function
self._score_name -> self.score_name
self._n_saved -> self.n_saved
self._ext -> self.ext
```
- The string formatting is not used when there is no filename_pattern provided
- When a filename_pattern is provided, we will use the string formatting to replace its values.
vfdev-5: @Joel-hanson yes I think your notes are correct. Two last points 
>  - The string formatting is not used when there is no filename_pattern provided
>  - When a filename_pattern is provided, we will use the string formatting to replace its values.

may be not clear how to implement. It could nice to have a single code for user pattern or our pattern. We can setup our default pattern `{filename_prefix}{name}_{global_step}{score_name}{score}.{ext}` at `__init__` if nothing provided. In `__call__` we can use something like 
```
    def _add_field(f, sep="_"):
        return "{}{}".format(f, sep) if len(f) > 0 else ""
```
for `"filename_prefix": add_field(kwargs.get("filename_prefix", ""))`, but certainly a specific processing of `{global_step}{score_name}{score}` is needed in order to have a good number of `_` etc.
Joel-hanson: @vfdev-5 @sdesrozis I have updated the PR with the required changes.

There are 2 things to note
1. I have not set any default filename pattern but works the same way as before, What I have done is to take the pattern only when we have a custom filename pattern assigned as an argument. @vfdev-5 I can change this to what you have suggested in the above comment to have default pattern `{filename_prefix}{name}_{global_step}{score_name}{score}.{ext}` if the current implementation doesn't meet the expectations.
2. What should we do when a user gives a filename pattern which doesn't have the right keys.
Example `{filename}{scores}.{ext}` this pattern has the keys `filename` and `scores` which we are not string formatting, this can cause a key error.
Will we mention this in the docs?
sdesrozis: @Joel-hanson keys should be described in docs and the list of handled keys must be fixed. 

Thank you again for your work!
sdesrozis: I played with the existing rules of checkpoint naming.

First, documentation states the rules (slightly modified)
```python
"""
- `filename_prefix` is the argument passed to the constructor,
- `name` is the key in `to_save` if a single object is to store, otherwise `name` is "checkpoint".
- `suffix` is composed as following `{global_step}_{score_name}={score}`.
   
Above `global_step` defined by the output of `global_step_transform` and `score` defined 
by the output of `score_function`.

By default, none of `score_function`, `score_name`, `global_step_transform` is defined, 
then suffix is setup by attached engine's current iteration. The filename will be
`{filename_prefix}_{name}_{engine.state.iteration}.{ext}`.

If defined a `score_function`, but without `score_name`, then suffix is defined by provided score.
The filename will be `{filename_prefix}_{name}_{score}.{ext}`. If `global_step_transform` is 
provided, then the filename will be `{filename_prefix}_{name}_{global_step}_{score}.{ext}`.

If defined `score_function` and `score_name`, then the filename will
be `{filename_prefix}_{name}_{score_name}={score}.{ext}`. 
If `global_step_transform` is provided, then the filename will be 
`{filename_prefix}_{name}_{global_step}_{score_name}={score}.{ext}`

For example, `score_name="neg_val_loss"` and `score_function` that returns `-loss` 
(as objects with highest scores will be retained), then saved filename will be
`{filename_prefix}_{name}_neg_val_loss=-0.1234.pt`.
"""
```

I wrote a tiny test which could help
```python
from typing import Callable, Optional

def test_pattern(
    filename_prefix: str = "",
    score_function: Optional[Callable] = None,
    score_name: Optional[str] = None,
    global_step_transform: Callable = None,
    filename_pattern: Optional[str] = None,
):
    # ignite rules are applied
    if filename_pattern is None:

        # filename_prefix will be included in pattern at the end
        
        if score_function is None and score_name is None and global_step_transform is None:
            # policy 1 `{filename_prefix}_{name}_{engine.state.iteration}.{ext}`
            filename_pattern = "{name}_%s.{ext}" % 203  # fake engine.state.iteration
        elif score_function and score_name is None:
            if global_step_transform:
                # policy 2a `{filename_prefix}_{name}_{global_step}_{score}.{ext}`
                filename_pattern = "{name}_{global_step}_{score}.{ext}"
            else:
                # policy 2b `{filename_prefix}_{name}_{global_step}_{score}.{ext}`
                filename_pattern = "{name}_{score}.{ext}"
        elif score_function and score_name:
            if global_step_transform:
                # policy 3a `{filename_prefix}_{name}_{global_step}_{score_name}={score}.{ext}`
                filename_pattern = "{name}_{global_step}_{score_name}={score}.{ext}"
            else:
                # policy 3b `{filename_prefix}_{name}_{global_step}_{score}.{ext}`
                filename_pattern = "{name}_{score_name}={score}.{ext}"
        else:
            raise Exception("weird case score_name is not None but score_function is None")
        
        # prefix is added to pattern if needed
        if filename_prefix:
            filename_pattern = "{filename_prefix}_" + filename_pattern

    # filename_pattern contains user specific pattern or ignite pattern

    # keys should be computed
    # compute score : fake call
    score = score_function() if score_function else None
    # compute global_step : fake call
    global_step = global_step_transform() if global_step_transform else None

    # dict of keys
    kwargs = {
        "filename_prefix": filename_prefix,
        "ext": "pt",
        "name": "checkpoint",
        "score_name": score_name,
        "score": score,
        "global_step": global_step
    }

    # pattern is applied wrt keys
    return filename_pattern.format(**kwargs)


if __name__ == '__main__':
    print(test_pattern())
    print(test_pattern("best"))
    print(test_pattern(score_function=lambda: 0.9999))
    print(test_pattern(score_function=lambda: 0.9999, global_step_transform=lambda: 12))
    print(test_pattern(score_function=lambda: 0.9999, score_name="acc"))
    print(test_pattern(score_function=lambda: 0.9999, score_name="acc", global_step_transform=lambda: 12))
    print(test_pattern("best", score_function=lambda: 0.9999))
    print(test_pattern("best", score_function=lambda: 0.9999, global_step_transform=lambda: 12))
    print(test_pattern("best", score_function=lambda: 0.9999, score_name="acc"))
    print(test_pattern("best", score_function=lambda: 0.9999, score_name="acc", global_step_transform=lambda: 12))

    pattern = "SAVE:{name}-{score_name}--{score}.pth"
    print(test_pattern("best", score_function=lambda: 0.9999, score_name="acc", global_step_transform=lambda: 12, filename_pattern=pattern))
```

Results are
```python
checkpoint_203.pt
best_checkpoint_203.pt
checkpoint_0.9999.pt
checkpoint_12_0.9999.pt
checkpoint_acc=0.9999.pt
checkpoint_12_acc=0.9999.pt
best_checkpoint_0.9999.pt
best_checkpoint_12_0.9999.pt
best_checkpoint_acc=0.9999.pt
best_checkpoint_12_acc=0.9999.pt
SAVE:checkpoint-acc--0.9999.pth
```

Hope that helps and no bug inside :) 
Joel-hanson: @sdesrozis Thank you so much for the docs and the test. I shall add it to this PR.
I have some doubt

1. The docstring which you have shared didn't have the first part which is 

```
This class stores a single file as a dictionary of provided objects to save.
The filename has the following structure: `{filename_prefix}_{name}_{suffix}.{ext}` where
``` 
is this intentional if so I will remove this from the docstring.

---
2. I have read the docstring and found that this is the only part which is updated

**from**
```
If defined a `score_function`, but without `score_name`, then suffix is defined by provided score.
The filename will be `{filename_prefix}_{name}_{global_step}_{score}.pt`.
```
**to**
```
If defined a `score_function`, but without `score_name`, then the suffix is defined by the provided score.
The filename will be `{filename_prefix}_{name}_{score}.{ext}`. If `global_step_transform` is provided,
then the filename will be `{filename_prefix}_{name}_{global_step}_{score}.{ext}`.
```
correct, this is just to confirm.

---
3. The current code has the checkpoint extension as `self.ext = ".pt"`, so when someone gives a filename pattern like `custom_{name}_{global_step}_{score}.{ext}` he will get it like with 2 dots in front of extension.
`current_checkpoint_acc_0.9999..pt`. So I shall change the default extension value to pt, ie `self.ext = "pt"` this will solve this issue.

sdesrozis: @Joel-hanson 

1. I extracted partially and I missed this part. The idea was to have rules in mind. Please, don't remove the missing part from docstring.
2. That is correct.
3. You are right. The dot should be removed. My idea was to have patterns written in the most understandable way. Maybe the current implementation fits perfectly the different cases but I was not sure... Feel free to use (and change) this snippet if it helps otherwise we could use it to check implementation. 
Joel-hanson: @sdesrozis @vfdev-5 I have updated the PR these latest changes
1. Updated the docstring as requested.
2. The dot in the extension is updated accordingly.
3. The tests for this feature is also added.
4. bug: I had missed the assignment of `name` in the filename_pattern, so I have fixed it.

The test function is a bit ugly and it's too long is there any suggestions or thoughts on this. Thanks, @sdesrozis I have used your test_pattern function for testing the default filenames of the checkpoints.
vfdev-5: Just to make you aware of this PR (https://github.com/pytorch/ignite/pull/1135/) and that here a filename is setup with a default pattern... Probably, that PR will be merged before this one. It would be nice to take into account it and modify `TrainsSaver` code too. 
Joel-hanson: > Just to make you aware of this PR (#1135) and that here a filename is setup with a default pattern... Probably, that PR will be merged before this one. It would be nice to take into account it and modify `TrainsSaver` code too.

I Happy to help @vfdev-5. I will look into it that as well and make the required changes.

@sdesrozis @vfdev-5 
I would like to note down what I have understood and what I have in mind.
1. I have to update the part which makes the filename for checkpoints (current version) with the helper function, as it is much more readable.
2. Have to update the test to have the expected values instead of the helper function.
3. Have to look into the issue which @vfdev-5 had reported to update `TrainsSaver` (I have not read it completely will post an update after going through the whole issue thread.
sdesrozis: Point 1. and 2. : perfect 👌🏻 

Point 3. : PR1135 will be merged first and use custom filename pattern. We must update wrt this pour 😉 
Joel-hanson: @sdesrozis @sdesrozis I have updated the code.

There are some doubts:
1. I have added the line `global_step = None` to have all the key values for formatting the filename_pattern is it ok to have None as its value when formatting. https://github.com/pytorch/ignite/pull/1127/files#diff-18cd1a3c06de70b0e34b8fcf32940b30R299 
2. @sdesrozis I have applied the following lines of code to the regular pattern feature as there is a situation where the global step function is present and no score name and no score function is present.
```
            elif global_step_transform:
                filename_pattern = "{name}_{global_step}.{ext}"
```
L287-L288 
3. I read some of the comments on PR # 1135. @vfdev-5 is the prerequisite that you listed is linked to the same implementation of a filename pattern that need to be implemented for 'TrainsSaver' is it? can you please elaborate on it.
sdesrozis: @Joel-hanson It looks good to me 😊 thank you!! Just one small comment to check.

I don’t have check PR # 1135 yet, I do it soon. @vfdev-5 might have some inputs.
Joel-hanson: @vfdev-5 I can see that the PR is merged can you elaborate on what is needs to be done? what I understood is linked to the same implementation of a filename pattern that needs to be implemented for 'TrainsSaver' is it?
vfdev-5: @Joel-hanson sorry for delay with this PR as we were occupied by current release.
vfdev-5: > @vfdev-5 I can see that the PR is merged can you elaborate on what is needs to be done? what I understood is linked to the same implementation of a filename pattern that needs to be implemented for 'TrainsSaver' is it?

Actually, there is nothing to adapt in `TrainsSaver`, in the begining I though that filename is recomposed in the code, but actually it is not the case. We can keep it as is.
Joel-hanson: @vfdev-5 @sdesrozis I have pushed the latest changes 🤞 .
- Have updated the function _get_filename_pattern to seetup_filename_pattern which is not public and static
- The arguments which were defined with typing were removed.
```
    @staticmethod
    def setup_filename_pattern(
        filename_prefix, with_score, with_score_name, with_global_step_transform, filename_pattern=None,
    ):
```
- Have updated the argument as with_score, with_score_name and with_global_step_transform to have the following values 
```
 with_score=self.score_function is None,
 with_score_name=self.score_name is None,
 with_global_step_transform=global_step is None,
```
sdesrozis: @Joel-hanson thank you! I don’t have any comment more than those from @vfdev-5 😊

Thank you again for your work !
Joel-hanson: @vfdev-5 I have updated the code accordingly which fixes the following.
1. The docstring for this new filename_pattern argument - https://github.com/pytorch/ignite/pull/1127#discussion_r449188777
2. The typing issue with the arguments and also the removal of filename_pattern from static function `setup_filename_pattern` https://github.com/pytorch/ignite/pull/1127#discussion_r446630655
3. The setup filename exception update https://github.com/pytorch/ignite/pull/1127#discussion_r446630877
4. The tests for the public function `setup_filename_pattern` https://github.com/pytorch/ignite/pull/1127#discussion_r446631082
vfdev-5: @Joel-hanson I modified the code and docs from my side.
sdesrozis: I fixed double quotations. The PR is ready for merge when CI will be ok.
vfdev-5: @sdesrozis actually, docs rendering with double quotes is better than with single quotes. I prefer to keep double quotes.
sdesrozis: 😱 I completely missed that! Very sorry, I revert!

Patch:
diff --git a/ignite/handlers/checkpoint.py b/ignite/handlers/checkpoint.py
index 6ebc152b..67e33002 100644
--- a/ignite/handlers/checkpoint.py
+++ b/ignite/handlers/checkpoint.py
@@ -93,6 +93,9 @@ class Checkpoint(Serializable):
             Default is None, global_step based on attached engine. If provided, uses function output as global_step.
             To setup global step from another engine, please use :meth:`~ignite.handlers.global_step_from_engine`.
         archived (bool, optional): Deprecated argument as models saved by ``torch.save`` are already compressed.
+        filename_pattern (str, optional): If ``filename_pattern`` is provided, this pattern will be used to render
+            checkpoint filenames. If the pattern is not defined, the default pattern would be used. See Note for
+            details.
         include_self (bool): Whether to include the `state_dict` of this object in the checkpoint. If `True`, then
             there must not be another object in ``to_save`` with key ``checkpointer``.
 
@@ -101,32 +104,52 @@ class Checkpoint(Serializable):
 
     Note:
         This class stores a single file as a dictionary of provided objects to save.
-        The filename has the following structure: `{filename_prefix}_{name}_{suffix}.{ext}` where
+        The filename is defined by ``filename_pattern`` and by default has the following
+        structure: ``{filename_prefix}_{name}_{suffix}.{ext}`` where
 
         - ``filename_prefix`` is the argument passed to the constructor,
         - `name` is the key in ``to_save`` if a single object is to store, otherwise `name` is "checkpoint".
-        - `suffix` is composed as following `{global_step}_{score_name}={score}`.
+        - `suffix` is composed as following ``{global_step}_{score_name}={score}``.
+
+    +----------------+------------+-----------------------+----------------------------------------------+
+    | score_function | score_name | global_step_transform |  suffix                                      |
+    +================+============+=======================+==============================================+
+    |      None      |   None     |        None           | ``{engine.state.iteration}``                 |
+    +----------------+------------+-----------------------+----------------------------------------------+
+    |       X        |   None     |        None           | ``{score}``                                  |
+    +----------------+------------+-----------------------+----------------------------------------------+
+    |       X        |   None     |         X             | ``{global_step}_{score}``                    |
+    +----------------+------------+-----------------------+----------------------------------------------+
+    |       X        |    X       |         X             | ``{global_step}_{score_name}={score}``       |
+    +----------------+------------+-----------------------+----------------------------------------------+
+    |      None      |   None     |         X             | ``{global_step}``                            |
+    +----------------+------------+-----------------------+----------------------------------------------+
+    |       X        |    X       |        None           | ``{score_name}={score}``                     |
+    +----------------+------------+-----------------------+----------------------------------------------+
+
+    Above `global_step` defined by the output of `global_step_transform` and `score` defined by the output
+    of `score_function`.
+
+    By default, none of ``score_function``, ``score_name``, ``global_step_transform`` is defined, then suffix is
+    setup by attached engine's current iteration. The filename will be
+    `{filename_prefix}_{name}_{engine.state.iteration}.{ext}`.
+
+    For example, ``score_name="neg_val_loss"`` and ``score_function`` that returns `-loss` (as objects with highest
+    scores will be retained), then saved filename will be ``{filename_prefix}_{name}_neg_val_loss=-0.1234.pt``.
 
-        Above `global_step` defined by the output of `global_step_transform` and `score` defined by the output
-        of `score_function`.
-
-        By default, none of ``score_function``, ``score_name``, ``global_step_transform`` is defined, then suffix is
-        setup by attached engine's current iteration. The filename will be
-        `{filename_prefix}_{name}_{engine.state.iteration}.{ext}`.
-
-        If only ``global_step_transform`` is defined, then suffix is setup using its return value.
-        The filename will be `{filename_prefix}_{name}_{global_step}.{ext}`.
-
-        If defined a ``score_function``, but without ``score_name``, then suffix is defined by provided score.
-        The filename will be `{filename_prefix}_{name}_{global_step}_{score}.pt`.
+    Note:
+        If ``filename_pattern`` is given, it will be used to render the filenames. ``filename_pattern`` is a string
+        that can contain ``{filename_prefix}``, ``{name}``, ``{score}``, ``{score_name}`` and ``{global_step}`` as
+        templates.
 
-        If defined ``score_function`` and ``score_name``, then the filename will
-        be `{filename_prefix}_{name}_{score_name}={score}.{ext}`. If ``global_step_transform`` is provided, then
-        the filename will be `{filename_prefix}_{name}_{global_step}_{score_name}={score}.{ext}`
+        For example, let ``filename_pattern="{global_step}-{name}-{score}.pt"`` then the saved filename will be
+        ``30000-checkpoint-94.pt``
 
-        For example, ``score_name="neg_val_loss"`` and ``score_function`` that returns `-loss` (as objects with highest
-        scores will be retained), then saved filename will be `{filename_prefix}_{name}_neg_val_loss=-0.1234.pt`.
+        **Warning:** Please, keep in mind that if filename collide with already used one to saved a checkpoint,
+        new checkpoint will not be stored. This means that filename like ``checkpoint.pt`` will be saved only once
+        and will not be overwritten by newer checkpoints.
 
+    Note:
         To get the last stored filename, handler exposes attribute ``last_checkpoint``:
 
         .. code-block:: python
@@ -143,8 +166,8 @@ class Checkpoint(Serializable):
 
     .. warning::
 
-        When running on TPUs, it should be run in all processes, otherwise application can get stuck on saving the
-        checkpoint.
+        When running on XLA devices, it should be run in all processes, otherwise application can get stuck on
+        saving the checkpoint.
 
         .. code-block:: python
 
@@ -219,6 +242,7 @@ class Checkpoint(Serializable):
         n_saved: Optional[int] = 1,
         global_step_transform: Callable = None,
         archived: bool = False,
+        filename_pattern: Optional[str] = None,
         include_self: bool = False,
     ):
 
@@ -254,14 +278,15 @@ class Checkpoint(Serializable):
             warnings.warn("Argument archived is deprecated and will be removed in 0.5.0")
 
         self.to_save = to_save
-        self._fname_prefix = filename_prefix + "_" if len(filename_prefix) > 0 else filename_prefix
+        self.filename_prefix = filename_prefix
         self.save_handler = save_handler
-        self._score_function = score_function
-        self._score_name = score_name
-        self._n_saved = n_saved
-        self._saved = []
-        self._ext = ".pt"
+        self.score_function = score_function
+        self.score_name = score_name
+        self.n_saved = n_saved
+        self.ext = "pt"
         self.global_step_transform = global_step_transform
+        self.filename_pattern = filename_pattern
+        self._saved = []
         self.include_self = include_self
 
     @property
@@ -271,27 +296,24 @@ class Checkpoint(Serializable):
         return self._saved[-1].filename
 
     def _check_lt_n_saved(self, or_equal=False):
-        if self._n_saved is None:
+        if self.n_saved is None:
             return True
-        return len(self._saved) < self._n_saved + int(or_equal)
+        return len(self._saved) < self.n_saved + int(or_equal)
 
     def __call__(self, engine: Engine) -> None:
 
-        suffix = ""
         global_step = None
         if self.global_step_transform is not None:
             global_step = self.global_step_transform(engine, engine.last_event_name)
-            suffix = "{}".format(global_step)
 
-        if self._score_function is not None:
-            priority = self._score_function(engine)
+        if self.score_function is not None:
+            priority = self.score_function(engine)
             if not isinstance(priority, numbers.Number):
                 raise ValueError("Output of score_function should be a number")
         else:
-            if global_step is not None:
-                priority = global_step
-            else:
-                priority = engine.state.get_event_attrib_value(Events.ITERATION_COMPLETED)
+            if global_step is None:
+                global_step = engine.state.get_event_attrib_value(Events.ITERATION_COMPLETED)
+            priority = global_step
 
         if self._check_lt_n_saved() or self._saved[0].priority < priority:
 
@@ -299,17 +321,6 @@ class Checkpoint(Serializable):
                 "{}".format(priority) if isinstance(priority, numbers.Integral) else "{:.4f}".format(priority)
             )
 
-            if self._score_name is not None:
-                if len(suffix) > 0:
-                    suffix += "_"
-                suffix = "{}{}={}".format(suffix, self._score_name, priority_str)
-            elif self._score_function is not None:
-                if len(suffix) > 0:
-                    suffix += "_"
-                suffix = "{}{}".format(suffix, priority_str)
-            elif len(suffix) == 0:
-                suffix = "{}".format(priority_str)
-
             checkpoint = self._setup_checkpoint()
 
             name = "checkpoint"
@@ -317,14 +328,33 @@ class Checkpoint(Serializable):
                 for k in checkpoint:
                     name = k
                 checkpoint = checkpoint[name]
-            filename = "{}{}_{}{}".format(self._fname_prefix, name, suffix, self._ext)
+
+            if self.filename_pattern is None:
+                filename_pattern = self.setup_filename_pattern(
+                    with_prefix=len(self.filename_prefix) > 0,
+                    with_score=self.score_function is not None,
+                    with_score_name=self.score_name is not None,
+                    with_global_step=global_step is not None,
+                )
+            else:
+                filename_pattern = self.filename_pattern
+
+            filename_dict = {
+                "filename_prefix": self.filename_prefix,
+                "ext": self.ext,
+                "name": name,
+                "score_name": self.score_name,
+                "score": priority_str if (self.score_function is not None) else None,
+                "global_step": global_step,
+            }
+            filename = filename_pattern.format(**filename_dict)
 
             if any(item.filename == filename for item in self._saved):
                 return
 
             metadata = {
-                "basename": "{}{}".format(self._fname_prefix, name),
-                "score_name": self._score_name,
+                "basename": "{}{}{}".format(self.filename_prefix, "_" * int(len(self.filename_prefix) > 0), name),
+                "score_name": self.score_name,
                 "priority": priority,
             }
 
@@ -353,6 +383,56 @@ class Checkpoint(Serializable):
             checkpoint[k] = obj.state_dict()
         return checkpoint
 
+    @staticmethod
+    def setup_filename_pattern(
+        with_prefix: bool = True, with_score: bool = True, with_score_name: bool = True, with_global_step: bool = True,
+    ) -> str:
+        """Helper method to get the default filename pattern for a checkpoint.
+
+        Args:
+            with_prefix (bool): If True, the ``filename_prefix`` is added to the filename pattern:
+                ``{filename_prefix}_{name}...``. Default, True.
+            with_score (bool): If True, ``score`` is added to the filename pattern: ``..._{score}.{ext}``.
+                Default, True. At least one of ``with_score`` and ``with_global_step`` should be True.
+            with_score_name (bool): If True, ``score_name`` is added to the filename pattern:
+                ``..._{score_name}={score}.{ext}``. If activated, argument ``with_score`` should be
+                also True, otherwise an error is raised. Default, True.
+            with_global_step (bool): If True, ``{global_step}`` is added to the
+                filename pattern: ``...{name}_{global_step}...``.
+                At least one of ``with_score`` and ``with_global_step`` should be True.
+
+        Example:
+
+            .. code-block:: python
+
+                from ignite.handlers import Checkpoint
+
+                filename_pattern = Checkpoint.setup_filename_pattern()
+
+                print(filename_pattern)
+                > "{filename_prefix}_{name}_{global_step}_{score_name}={score}.{ext}"
+        """
+        filename_pattern = "{name}"
+
+        if not (with_global_step or with_score):
+            raise ValueError("At least one of with_score and with_global_step should be True.")
+
+        if with_global_step:
+            filename_pattern += "_{global_step}"
+
+        if with_score_name and with_score:
+            filename_pattern += "_{score_name}={score}"
+        elif with_score:
+            filename_pattern += "_{score}"
+        elif with_score_name:
+            raise ValueError("If with_score_name is True, with_score should be also True")
+
+        if with_prefix:
+            filename_pattern = "{filename_prefix}_" + filename_pattern
+
+        filename_pattern += ".{ext}"
+        return filename_pattern
+
     @staticmethod
     def _check_objects(objs: Mapping, attr: str) -> None:
         for k, obj in objs.items():


Test patch:
diff --git a/tests/ignite/handlers/test_checkpoint.py b/tests/ignite/handlers/test_checkpoint.py
index 9d6a779b..0369cc27 100644
--- a/tests/ignite/handlers/test_checkpoint.py
+++ b/tests/ignite/handlers/test_checkpoint.py
@@ -1211,6 +1211,167 @@ def test_distrib_single_device_xla_nprocs(xmp_executor, dirname):
     xmp_executor(_test_tpu_saves_to_cpu_nprocs, args=(dirname,), nprocs=n)
 
 
+def test_checkpoint_filename_pattern():
+    def _test(
+        to_save,
+        filename_prefix="",
+        score_function=None,
+        score_name=None,
+        global_step_transform=None,
+        filename_pattern=None,
+    ):
+        save_handler = MagicMock(spec=BaseSaveHandler)
+
+        checkpointer = Checkpoint(
+            to_save,
+            save_handler=save_handler,
+            filename_prefix=filename_prefix,
+            score_function=score_function,
+            score_name=score_name,
+            global_step_transform=global_step_transform,
+            filename_pattern=filename_pattern,
+        )
+
+        trainer = Engine(lambda e, b: None)
+        trainer.state = State(epoch=12, iteration=203, score=0.9999)
+
+        checkpointer(trainer)
+        return checkpointer.last_checkpoint
+
+    model = DummyModel()
+    to_save = {"model": model}
+
+    assert _test(to_save) == "model_203.pt"
+    assert _test(to_save, "best") == "best_model_203.pt"
+    assert _test(to_save, score_function=lambda e: e.state.score) == "model_0.9999.pt"
+
+    res = _test(to_save, score_function=lambda e: e.state.score, global_step_transform=lambda e, _: e.state.epoch)
+    assert res == "model_12_0.9999.pt"
+
+    assert _test(to_save, score_function=lambda e: e.state.score, score_name="acc") == "model_acc=0.9999.pt"
+
+    res = _test(
+        to_save,
+        score_function=lambda e: e.state.score,
+        score_name="acc",
+        global_step_transform=lambda e, _: e.state.epoch,
+    )
+    assert res == "model_12_acc=0.9999.pt"
+
+    assert _test(to_save, "best", score_function=lambda e: e.state.score) == "best_model_0.9999.pt"
+
+    res = _test(
+        to_save, "best", score_function=lambda e: e.state.score, global_step_transform=lambda e, _: e.state.epoch
+    )
+    assert res == "best_model_12_0.9999.pt"
+
+    res = _test(to_save, "best", score_function=lambda e: e.state.score, score_name="acc")
+    assert res == "best_model_acc=0.9999.pt"
+
+    res = _test(
+        to_save,
+        "best",
+        score_function=lambda e: e.state.score,
+        score_name="acc",
+        global_step_transform=lambda e, _: e.state.epoch,
+    )
+    assert res == "best_model_12_acc=0.9999.pt"
+
+    pattern = "chk-{name}--{global_step}.{ext}"
+    assert _test(to_save, to_save, filename_pattern=pattern) == "chk-model--203.pt"
+    pattern = "chk-{filename_prefix}--{name}--{global_step}.{ext}"
+    assert _test(to_save, "best", filename_pattern=pattern) == "chk-best--model--203.pt"
+    pattern = "chk-{name}--{score}.{ext}"
+    assert _test(to_save, score_function=lambda e: e.state.score, filename_pattern=pattern) == "chk-model--0.9999.pt"
+    pattern = "{global_step}-{name}-{score}.chk.{ext}"
+    res = _test(
+        to_save,
+        score_function=lambda e: e.state.score,
+        global_step_transform=lambda e, _: e.state.epoch,
+        filename_pattern=pattern,
+    )
+    assert res == "12-model-0.9999.chk.pt"
+
+    pattern = "chk-{name}--{score_name}--{score}.{ext}"
+    res = _test(to_save, score_function=lambda e: e.state.score, score_name="acc", filename_pattern=pattern)
+    assert res == "chk-model--acc--0.9999.pt"
+
+    pattern = "chk-{name}-{global_step}-{score_name}-{score}.{ext}"
+    res = _test(
+        to_save,
+        score_function=lambda e: e.state.score,
+        score_name="acc",
+        global_step_transform=lambda e, _: e.state.epoch,
+        filename_pattern=pattern,
+    )
+    assert res == "chk-model-12-acc-0.9999.pt"
+
+    pattern = "{filename_prefix}-{name}-{score}.chk"
+    res = _test(to_save, "best", score_function=lambda e: e.state.score, filename_pattern=pattern)
+    assert res == "best-model-0.9999.chk"
+
+    pattern = "resnet-{filename_prefix}-{name}-{global_step}-{score}.chk"
+    res = _test(
+        to_save,
+        "best",
+        score_function=lambda e: e.state.score,
+        global_step_transform=lambda e, _: e.state.epoch,
+        filename_pattern=pattern,
+    )
+    assert res == "resnet-best-model-12-0.9999.chk"
+
+    pattern = "{filename_prefix}-{name}-{score_name}-{score}.chk"
+    res = _test(to_save, "best", score_function=lambda e: e.state.score, score_name="acc", filename_pattern=pattern)
+    assert res == "best-model-acc-0.9999.chk"
+
+    pattern = "{global_step}-{filename_prefix}-{name}-{score_name}-{score}"
+    res = _test(
+        to_save,
+        "best",
+        score_function=lambda e: e.state.score,
+        score_name="acc",
+        global_step_transform=lambda e, _: e.state.epoch,
+        filename_pattern=pattern,
+    )
+    assert res == "12-best-model-acc-0.9999"
+
+    pattern = "SAVE:{name}-{score_name}-{score}.pth"
+    res = _test(
+        to_save,
+        "best",
+        score_function=lambda e: e.state.score,
+        score_name="acc",
+        global_step_transform=lambda e, _: e.state.epoch,
+        filename_pattern=pattern,
+    )
+
+    assert res == "SAVE:model-acc-0.9999.pth"
+
+    pattern = "{global_step}-chk-{filename_prefix}-{name}-{score_name}-{score}.{ext}"
+    assert _test(to_save, filename_pattern=pattern) == "203-chk--model-None-None.pt"
+
+    with pytest.raises(KeyError, match=r"random_key"):
+        pattern = "SAVE:{random_key}.{ext}"
+        _test(to_save, filename_pattern=pattern)
+
+
+def test_setup_filename_pattern():
+    # default filename pattern
+    assert Checkpoint.setup_filename_pattern() == "{filename_prefix}_{name}_{global_step}_{score_name}={score}.{ext}"
+
+    assert Checkpoint.setup_filename_pattern(False) == "{name}_{global_step}_{score_name}={score}.{ext}"
+    assert Checkpoint.setup_filename_pattern(False, False, False) == "{name}_{global_step}.{ext}"
+    assert Checkpoint.setup_filename_pattern(False, True, False) == "{name}_{global_step}_{score}.{ext}"
+    assert Checkpoint.setup_filename_pattern(False, True, False, False) == "{name}_{score}.{ext}"
+    assert Checkpoint.setup_filename_pattern(False, True, True, False) == "{name}_{score_name}={score}.{ext}"
+
+    with pytest.raises(ValueError, match=r"At least one of with_score and with_global_step should be True."):
+        Checkpoint.setup_filename_pattern(False, False, False, False)
+
+    with pytest.raises(ValueError, match=r"If with_score_name is True, with_score should be also True"):
+        Checkpoint.setup_filename_pattern(True, False, True, True)
+
+
 def _setup_checkpoint():
     save_handler = MagicMock(spec=BaseSaveHandler)
     model = DummyModel()