Repository: stable-diffusion-webui
Issue: [Feature Request]: Set cutoff tokens in prompt at will using the "." symbol and special CLIP tokens (this is easy to do)
Labels: enhancement

### What would your feature do ?

I'm developing an extension and discovered something strange in the configuration with the CLIPTokenizer on Automatic1111 for an SDXL model i downloaded from civitai.

I was running CLIPtokenizer.encode(text) on a given input string  and the tokenizer seems to append cutoff tokens |start-of-text| and |end-of-text| between every token.

![blub1](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130230016/598edf9c-4962-4958-bec7-f214e1fd03ab)

Here is a a printout of Automatic1111/modules/shared/sd_model.cond_stage_model.wrapped:

'####model.cond_stage_model.embedders.wrapped[0]####'
{'_backward_hooks': OrderedDict(),
 '_backward_pre_hooks': OrderedDict(),
 '_buffers': OrderedDict(),
 '_forward_hooks': OrderedDict(),
 '_forward_hooks_with_kwargs': OrderedDict(),
 '_forward_pre_hooks': OrderedDict(),
 '_forward_pre_hooks_with_kwargs': OrderedDict(),
 '_input_key': 'txt',
 '_is_full_backward_hook': None,
 '_is_trainable': False,
 '_load_state_dict_post_hooks': OrderedDict(),
 '_load_state_dict_pre_hooks': OrderedDict(),
 '_modules': OrderedDict([('transformer',
                           CLIPTextModel(
  (text_model): CLIPTextTransformer(
    (embeddings): CLIPTextEmbeddings(
      (token_embedding): EmbeddingsWithFixes(
        (wrapped): Embedding(49408, 768)
      )
      (position_embedding): Embedding(77, 768)
    )
    (encoder): CLIPEncoder(
      (layers): ModuleList(
        (0-11): 12 x CLIPEncoderLayer(
          (self_attn): CLIPAttention(
            (k_proj): Linear(in_features=768, out_features=768, bias=True)
            (v_proj): Linear(in_features=768, out_features=768, bias=True)
            (q_proj): Linear(in_features=768, out_features=768, bias=True)
            (out_proj): Linear(in_features=768, out_features=768, bias=True)
          )
          (layer_norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (mlp): CLIPMLP(
            (activation_fn): QuickGELUActivation()
            (fc1): Linear(in_features=768, out_features=3072, bias=True)
            (fc2): Linear(in_features=3072, out_features=768, bias=True)
          )
          (layer_norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
        )
      )
    )
    (final_layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
  )
))]),
 '_non_persistent_buffers_set': set(),
 '_parameters': OrderedDict(),
 '_state_dict_hooks': OrderedDict(),
 '_state_dict_pre_hooks': OrderedDict(),
 '_ucg_rate': 0.0,
 'device': 'cuda',
 'layer': 'hidden',
 'layer_idx': 11,
 'legacy_ucg_val': None,
 'max_length': 77,
 'return_pooled': False,

 '**tokenizer': CLIPTokenizer**(name_or_path='openai/clip-vit-large-patch14', vocab_size=49408, model_max_length=77, is_fast=False, padding_side='right', truncation_side='right', special_tokens={'bos_token': AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), 'eos_token': AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), 'unk_token': AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), 'pad_token': '
'}, clean_up_tokenization_spaces=True),

 'train': <function disabled_train at 0x7a5f1d845870>,
 'training': True}
'####END OF model.cond_stage_model.embedders.wrapped[0]####'

I realize this problem arose because I tokenized the text piece by piece in my code. I expect (or well , I sincerly hope ) that this configuration exists in A1111 because  the tokenizer tokenizes the prompt "all-at-once" rather then "word-by-word" as I do currently. 

It would be pretty bad for prompt comprehension if cutoff tokens were placed as shown in the first image in A1111. But I trust that is under control. 

Anyway , I fixed this issue in my extension by filtering out the cutoff tokens from the configured A1111 tokenizer and letting the user assign the symbol "<" and ">" to place a |start-of-text| and |end-of-text| token in the output. This made prompt comprehension much better and was very convenient.

![cutoff1](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130230016/ac019050-b95c-45be-a6e3-7f6753bd33fd)

So, I think this would be a really neat feature to have in Automatic1111 in the prompt box. You could let users write a special symbol in the prompt box like the dot "." symbol (or something else) and have the tokenizer interpret that as a cutoff token. 

EDIT: Turns out this feature also exists within SD 1.5

![Cutoff SD 1 5](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130230016/3f7e0966-d511-4d72-a5d2-68ceedbaa271)

### Proposed workflow

When using SDXL , SD2 or SD1.5 model:
1. Write "." in the prompt to place |start-of-text| token there (token ID# is  49406) (context dependent)
2. Write "." in the prompt to place |end-of-text| token (token ID# is  49407)
(also context dependent)

TLDR:
You want the prompt "A . B C  . D " to become tokenized to 
"< sot > A < eot > < sot > B C < eot > < sot > D < eot > "

where < sot > = |start-of-text| token 
and < eot > = |end-of-text| token

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits