Trained for 0 epochs and 1000 steps.

Trained with datasets ['text-embeds', 'sfwbooru-crop']
Learning rate 1.0, batch size 4, and 1 gradient accumulation steps.
Used DDPM noise scheduler for training with epsilon prediction type and rescaled_betas_zero_snr=False
Using 'trailing' timestep spacing.
Base model: stabilityai/stable-diffusion-3.5-medium
VAE: None

.gitattributes

@@ -89,3 +89,4 @@ assets/image_6_0.png filter=lfs diff=lfs merge=lfs -text
 assets/image_7_0.png filter=lfs diff=lfs merge=lfs -text
 assets/image_8_0.png filter=lfs diff=lfs merge=lfs -text
 assets/image_9_0.png filter=lfs diff=lfs merge=lfs -text
+training_state-sfwbooru-crop.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -329,13 +329,13 @@ You may reuse the base model text encoder for inference.
 ## Training settings
 
 - Training epochs: 0
-- Training steps: 100
+- Training steps: 1000
 - Learning rate: 1.0
   - Learning rate schedule: constant
   - Warmup steps: 500
-- Max grad norm: 0.01
-- Effective batch size: 6
-  - Micro-batch size: 6
+- Max grad norm: 0.0
+- Effective batch size: 4
+  - Micro-batch size: 4
   - Gradient accumulation steps: 1
   - Number of GPUs: 1
 - Gradient checkpointing: True
@@ -358,15 +358,6 @@ You may reuse the base model text encoder for inference.
 - Crop style: random
 - Crop aspect: square
 - Used for regularisation data: No
-### sfwbooru
-- Repeats: 0
-- Total number of images: 469191
-- Total number of aspect buckets: 70
-- Resolution: 1.048576 megapixels
-- Cropped: False
-- Crop style: None
-- Crop aspect: None
-- Used for regularisation data: No
 
 
 ## Inference
@@ -397,3 +388,11 @@ image.save("output.png", format="PNG")
 
 
 
+## Exponential Moving Average (EMA)
+
+SimpleTuner generates a safetensors variant of the EMA weights and a pt file.
+
+The safetensors file is intended to be used for inference, and the pt file is for continuing finetuning.
+
+The EMA model may provide a more well-rounded result, but typically will feel undertrained compared to the full model as it is a running decayed average of the model weights.
+

optimizer.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6274f068dddc60934117c2fe0fdc19194d7c7ef632ebb79d6656a6bcc0954f50
+size 5313950865

random_states_0.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3396fb45d05f866cede6ac17b1016a0ea31e7c16413cb4924c4312c77f29a03
+size 14344

training_state-sfwbooru-crop.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82435ef38f168d82561ef3ed42940e3cfb7233af3dd51d6d2ff06ef22b5c05f0
+size 14740962

training_state.json

@@ -0,0 +1 @@
+{"global_step": 1000, "epoch_step": 1000, "epoch": 1, "exhausted_backends": [], "repeats": {}}

transformer/diffusion_pytorch_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6f58e2d4094eaf093c27eed83243547d8d01510fcac23e0a80d88049a31f7a1a
+oid sha256:f14fdae48f7932fd7481a2373d8507ce9d451295dcd044d2ca98188975c1baf8
 size 4939433672

transformer_ema/config.json

@@ -0,0 +1,39 @@
+{
+  "_class_name": "SD3Transformer2DModel",
+  "_diffusers_version": "0.32.0.dev0",
+  "_name_or_path": "stabilityai/stable-diffusion-3.5-medium",
+  "attention_head_dim": 64,
+  "caption_projection_dim": 1536,
+  "decay": 0.995,
+  "dual_attention_layers": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12
+  ],
+  "in_channels": 16,
+  "inv_gamma": 1.0,
+  "joint_attention_dim": 4096,
+  "min_decay": 0.0,
+  "num_attention_heads": 24,
+  "num_layers": 24,
+  "optimization_step": 1000,
+  "out_channels": 16,
+  "patch_size": 2,
+  "pooled_projection_dim": 2048,
+  "pos_embed_max_size": 384,
+  "power": 0.6666666666666666,
+  "qk_norm": "rms_norm",
+  "sample_size": 128,
+  "update_after_step": 0,
+  "use_ema_warmup": false
+}

transformer_ema/diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94a7ec3026986d8026933cea83042ad4959ffde3cb2da707557b65f9747ff43a
+size 9878760864

transformer_ema/ema_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd018af71196ba782d63cca6bb9fd86be0fd14ba2ca9d2e37aefa5f3bb5fbf12
+size 4486612394