Repository: stable-diffusion-webui
Issue: [Feature Request]: Fix/Enhancement for last step and/or discard next to last sigma
Labels: enhancement

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What would your feature do ?

For longest time I've had this issue on A1111, and as far as I know still isn't resolved...

I've had the last steps of generation ruin a lot of generations, where preview looks perfect, then splats it at last second, or so it looks like watching preview. Recently seems a bit more focused on faces but can still be whole image on teetering CFG settings.

after finally figuring it out, this is only on certain samplers, all SDMs/SDEs I think, maybe Eular too. (possibly exception of karras variations.)

trying quick fixes like the current discard next to last sigma, and using extensions like anti-burn to force/brute-force discarding last steps have not helped avoid as workarounds. The fix I've done feels more like a solution than a workaround anyway.

Fixed it by researching previous solutions that don't seem to be implemented at all, but has been researched in the past, a previous PR (https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12349/files) even supplied a prior straight forward fix, but is outdated so thought I'd post updated instructions for that PR as it took me a bit to figure out and I imagine could help someone.

I also think this should just be added to the code for discard next to last sigma, as it makes that setting now perform exactly what I expect it to.

### Proposed workflow

SD-Webui/Modules/
====================================================================================
SD_samplers_kdiffusion.py

------------------------------------------------------------------------------------

Line 18

```
    ('Euler', 'sample_euler', ['k_euler'], {}),
    ('LMS', 'sample_lms', ['k_lms'], {'discard_next_to_last_sigma': True, 'use_last_stored_latent': True}), #<<<+
    ('Heun', 'sample_heun', ['k_heun'], {"second_order": True}),

```


------------------------------------------------------------------------------------

Lines 230-234


```
 if self.config.options.get('solver_type', None) == 'heun':
     extra_params_kwargs['solver_type'] = 'heun'

#(ALL___VVV+VVV___ALL)
 use_last_stored_latent = self.config is not None and self.config.options.get('use_last_stored_latent', False)
 if opts.use_last_stored_latent and not use_last_stored_latent:
     use_last_stored_latent = True
     p.extra_generation_params["Use last stored latent"] = True
 self.last_latent = x if use_last_stored_latent == True else "" #<<<~/- (modify or replace existing line)
#(ALL___^^^+^^^___ALL)
 self.sampler_extra_args = {
     'cond': conditioning,
     'image_cond': image_conditioning,
     'uncond': unconditional_conditioning,
     'cond_scale': p.cfg_scale,
     's_min_uncond': self.s_min_uncond
```

------------------------------------------------------------------------------------

Lines 179-183

```

        if self.config.options.get('solver_type', None) == 'heun':
            extra_params_kwargs['solver_type'] = 'heun'

        self.model_wrap_cfg.init_latent = x
#(ALL___VVV+VVV___ALL)
 	use_last_stored_latent = self.config is not None and self.config.options.get('use_last_stored_latent', False)
 	if opts.use_last_stored_latent and not use_last_stored_latent:
     		use_last_stored_latent = True
     		p.extra_generation_params["Use last stored latent"] = True
 		self.last_latent = x if use_last_stored_latent == True else "" #<<<~/- (modify or replace existing line)
#(ALL___^^^+^^^___ALL)
        self.sampler_extra_args = {
            'cond': conditioning,
            'image_cond': image_conditioning,
            'uncond': unconditional_conditioning,
            'cond_scale': p.cfg_scale,
            's_min_uncond': self.s_min_uncond
```

------------------------------------------------------------------------------------

SD_samplers_timesteps.py

------------------------------------------------------------------------------------

Line 155 & 159

```
        if 'timesteps' in parameters:
            extra_params_kwargs['timesteps'] = timesteps

#(ALL___VVV+VVV___ALL)
        use_last_stored_latent = self.config is not None and self.config.options.get('use_last_stored_latent', False)
        if opts.use_last_stored_latent and not use_last_stored_latent:
            use_last_stored_latent = True
            p.extra_generation_params["Use last stored latent"] = True
        self.last_latent = x if use_last_stored_latent == True else ""  #<<<~/- (modify or replace existing line)
#(ALL___^^^+^^^___ALL)
        self.sampler_extra_args = {
            'cond': conditioning,
            'image_cond': image_conditioning,
            'uncond': unconditional_conditioning,
            'cond_scale': p.cfg_scale,
            's_min_uncond': self.s_min_uncond
```

------------------------------------------------------------------------------------

Line 125 & 129

------------------------------------------------------------------------------------

```
        if 'is_img2img' in parameters:
            extra_params_kwargs['is_img2img'] = True          
            

        self.model_wrap_cfg.init_latent = x
#(ALL___VVV+VVV___ALL)
        use_last_stored_latent = self.config is not None and self.config.options.get('use_last_stored_latent', False)
        if opts.use_last_stored_latent and not use_last_stored_latent:
            use_last_stored_latent = True
            p.extra_generation_params["Use last stored latent"] = True
        self.last_latent = x if use_last_stored_latent == True else ""  #<<<~/- (modify or replace existing line)
#(ALL___^^^+^^^___ALL)
        self.sampler_extra_args = {
            'cond': conditioning,
            'image_cond': image_conditioning,
            'uncond': unconditional_conditioning,
            'cond_scale': p.cfg_scale,
            's_min_uncond': self.s_min_uncond
```

------------------------------------------------------------------------------------

Shared_Options.py

------------------------------------------------------------------------------------

Line-352

```
    'always_discard_next_to_last_sigma': OptionInfo(False, "Always discard next-to-last sigma", infotext='Discard penultimate sigma').link("PR", "https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/6044"),
    'use_last_stored_latent': OptionInfo(False, "Always try to use the last stored latent").info("Used by default for LMS (non-Karras) to work around an issue with k-diffusion").link("PR", "https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12349"), #<<<+
    'sgm_noise_multiplier': OptionInfo(False, "SGM noise multiplier", infotext='SGM noise multplier').link("PR", "https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12818").info("Match initial noise to official SDXL implementation - only useful for reproducing images"),
```

------------------------------------------------------------------------------------

SD-Webui/scripts/
===================================================================================

xyz_grid.py

------------------------------------------------------------------------------------

Line 268

```
    AxisOption("Token merging ratio high-res", float, apply_override('token_merging_ratio_hr')),
    AxisOption("Always discard next-to-last sigma", str, apply_override('always_discard_next_to_last_sigma', boolean=True), choices=boolean_choice(reverse=True)), #<<<+
    AxisOption("Use last stored latent", str, apply_override('use_last_stored_latent', boolean=True), choices=boolean_choice(reverse=True)),
```

===================================================================================
!UPDATE!
------------------------------------------------------------------------------------
While I did see an immediate improvement using the above code adjustments,, still had a weird bright spot, instead of whole face or picture, but always a spot on 3/4 of generations on most samplers.
After further investigating seems multiple issues must of been going on, as that spot is caused by controlnet, and it seems to be the culprit for targeting the face so much (that's where the spot always is and eyes have been ROUGH lately).
weird thing is, this is without it enabled or having hardly any models loaded on a pretty fresh install... gonna overhaul mine, keep in mind, something weird like this can happen as well.

also, overhauled the above code suggestion, more sensible I think now.

### Additional information

Optional: Can remove the following if desired
```
#<<<+
#<<<~/-
#(All___VVV+VVV___ALL)
```
Just used to specify and document changed lines as extra lines are provided in case of different/updated versions changing line numbers; hopefully I commented them all and correctly, if so, shouldn't matter either way.

I can't python to save my life, not on my own, so the information provided from other(s) are much valued in figuring this out, all I really had to do was modernize to current version and slightly restructure the code from a previous version discussions.
Solution credits:
[bosbrand](https://github.com/bosbrand) https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/7244
[catboxanon](https://github.com/catboxanon) https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12349/files

Also, I don't know python, not really, so if my changes or suggestion does a dumb please feel free to point out the correct proper! :)