Repository: huggingface/transformers
Pull request: Fix inaccurate train_tokens_per_second when resuming from checkpoint

# Fix inaccurate `train_tokens_per_second` when resuming from checkpoint

## What does this PR do?
This PR addresses the issue where `train_tokens_per_second` calculates incorrectly when resuming training from a checkpoint. 

### Problem Background
Previously, `speed_metrics` used the **cumulative total token count** (`self.state.num_input_tokens_seen`) to compute training speed. However, when resuming from a checkpoint:
1. The cumulative token count includes tokens processed in prior sessions.
2. `start_time` is reset to the current session’s start time.
This mismatch led to abnormally high initial `train_tokens_per_second` values (as the cumulative tokens were divided by a short new time window), which gradually decreased but never accurately reflected the current session’s real throughput (reported in #40560).


### Solution
To ensure speed metrics reflect only the current training session:
1. **Capture baseline token count**: In the `_inner_training_loop` method (right after `start_time = time.time()`), save the initial token count of the current session as `initial_num_input_tokens_seen_for_session`.
2. **Calculate session-specific token increment**: When updating logs with `speed_metrics`, compute the number of tokens processed *in the current session* (via `self.state.num_input_tokens_seen - initial_num_input_tokens_seen_for_session`), and pass this increment to `speed_metrics` instead of the cumulative total.


This change ensures `train_tokens_per_second` consistently represents the real-time throughput of the active training session—whether starting fresh or resuming from a checkpoint.

Fixes #40560


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.  
  Links: #40560
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and here are tips on formatting docstrings.  
- [ ] Did you write any new necessary tests?  


## Who can review?


Changed files:
filename: src/transformers/trainer.py
patch: @@ -2516,6 +2516,7 @@ def _inner_training_loop(
 
         self.state.epoch = 0
         start_time = time.time()
+        self.initial_num_input_tokens_seen_for_session = self.state.num_input_tokens_seen
         epochs_trained = 0
         steps_trained_in_current_epoch = 0
 
@@ -3772,7 +3773,10 @@ def log(self, logs: dict[str, float], start_time: Optional[float] = None) -> Non
         if self.args.include_num_input_tokens_seen != "no":
             logs["num_input_tokens_seen"] = self.state.num_input_tokens_seen
             if start_time is not None:
-                logs.update(speed_metrics("train", start_time, num_tokens=self.state.num_input_tokens_seen))
+                current_session_num_tokens = (
+                    self.state.num_input_tokens_seen - self.initial_num_input_tokens_seen_for_session
+                )
+                logs.update(speed_metrics("train", start_time, num_tokens=current_session_num_tokens))
 
         output = {**logs, **{"step": self.state.global_step}}
         self.state.log_history.append(output)