| 10/15/2023 18:37:47 - INFO - app - Distributed environment: NO | |
| Num processes: 1 | |
| Process index: 0 | |
| Local process index: 0 | |
| Device: cuda | |
| Mixed precision type: no | |
| 10/15/2023 18:37:52 - INFO - app - Num examples = 5 | |
| 10/15/2023 18:37:52 - INFO - app - Num batches each epoch = 2 | |
| 10/15/2023 18:37:52 - INFO - app - Num Epochs = 1000 | |
| 10/15/2023 18:37:52 - INFO - app - Instantaneous batch size per device = 4 | |
| 10/15/2023 18:37:52 - INFO - app - Total train batch size (w. parallel, distributed & accumulation) = 4 | |
| 10/15/2023 18:37:52 - INFO - app - Gradient Accumulation steps = 1 | |
| 10/15/2023 18:37:52 - INFO - app - Training Steps = 2000 | |
| 10/15/2023 18:37:53 - INFO - app - STEP = 1 | |
| 10/15/2023 18:37:55 - INFO - app - STEP = 2 | |
| 10/15/2023 18:37:56 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:38:21 - INFO - app - STEP = 3 | |
| 10/15/2023 18:38:23 - INFO - app - STEP = 4 | |
| 10/15/2023 18:38:24 - INFO - app - STEP = 5 | |
| 10/15/2023 18:38:25 - INFO - app - STEP = 6 | |
| 10/15/2023 18:38:27 - INFO - app - STEP = 7 | |
| 10/15/2023 18:38:28 - INFO - app - STEP = 8 | |
| 10/15/2023 18:38:29 - INFO - app - STEP = 9 | |
| 10/15/2023 18:38:31 - INFO - app - STEP = 10 | |
| 10/15/2023 18:38:32 - INFO - app - STEP = 11 | |
| 10/15/2023 18:38:33 - INFO - app - STEP = 12 | |
| 10/15/2023 18:38:34 - INFO - app - STEP = 13 | |
| 10/15/2023 18:38:36 - INFO - app - STEP = 14 | |
| 10/15/2023 18:38:37 - INFO - app - STEP = 15 | |
| 10/15/2023 18:38:38 - INFO - app - STEP = 16 | |
| 10/15/2023 18:38:39 - INFO - app - STEP = 17 | |
| 10/15/2023 18:38:41 - INFO - app - STEP = 18 | |
| 10/15/2023 18:38:42 - INFO - app - STEP = 19 | |
| 10/15/2023 18:38:43 - INFO - app - STEP = 20 | |
| 10/15/2023 18:38:45 - INFO - app - STEP = 21 | |
| 10/15/2023 18:38:46 - INFO - app - STEP = 22 | |
| 10/15/2023 18:38:47 - INFO - app - STEP = 23 | |
| 10/15/2023 18:38:48 - INFO - app - STEP = 24 | |
| 10/15/2023 18:38:50 - INFO - app - STEP = 25 | |
| 10/15/2023 18:38:51 - INFO - app - STEP = 26 | |
| 10/15/2023 18:38:52 - INFO - app - STEP = 27 | |
| 10/15/2023 18:38:54 - INFO - app - STEP = 28 | |
| 10/15/2023 18:38:55 - INFO - app - STEP = 29 | |
| 10/15/2023 18:38:56 - INFO - app - STEP = 30 | |
| 10/15/2023 18:38:58 - INFO - app - STEP = 31 | |
| 10/15/2023 18:38:59 - INFO - app - STEP = 32 | |
| 10/15/2023 18:39:00 - INFO - app - STEP = 33 | |
| 10/15/2023 18:39:02 - INFO - app - STEP = 34 | |
| 10/15/2023 18:39:03 - INFO - app - STEP = 35 | |
| 10/15/2023 18:39:04 - INFO - app - STEP = 36 | |
| 10/15/2023 18:39:05 - INFO - app - STEP = 37 | |
| 10/15/2023 18:39:07 - INFO - app - STEP = 38 | |
| 10/15/2023 18:39:08 - INFO - app - STEP = 39 | |
| 10/15/2023 18:39:09 - INFO - app - STEP = 40 | |
| 10/15/2023 18:39:10 - INFO - app - STEP = 41 | |
| 10/15/2023 18:39:12 - INFO - app - STEP = 42 | |
| 10/15/2023 18:39:13 - INFO - app - STEP = 43 | |
| 10/15/2023 18:39:14 - INFO - app - STEP = 44 | |
| 10/15/2023 18:39:16 - INFO - app - STEP = 45 | |
| 10/15/2023 18:39:17 - INFO - app - STEP = 46 | |
| 10/15/2023 18:39:18 - INFO - app - STEP = 47 | |
| 10/15/2023 18:39:20 - INFO - app - STEP = 48 | |
| 10/15/2023 18:39:21 - INFO - app - STEP = 49 | |
| 10/15/2023 18:39:22 - INFO - app - STEP = 50 | |
| 10/15/2023 18:39:23 - INFO - app - STEP = 51 | |
| 10/15/2023 18:39:25 - INFO - app - STEP = 52 | |
| 10/15/2023 18:39:26 - INFO - app - STEP = 53 | |
| 10/15/2023 18:39:27 - INFO - app - STEP = 54 | |
| 10/15/2023 18:39:29 - INFO - app - STEP = 55 | |
| 10/15/2023 18:39:30 - INFO - app - STEP = 56 | |
| 10/15/2023 18:39:31 - INFO - app - STEP = 57 | |
| 10/15/2023 18:39:33 - INFO - app - STEP = 58 | |
| 10/15/2023 18:39:34 - INFO - app - STEP = 59 | |
| 10/15/2023 18:39:35 - INFO - app - STEP = 60 | |
| 10/15/2023 18:39:36 - INFO - app - STEP = 61 | |
| 10/15/2023 18:39:38 - INFO - app - STEP = 62 | |
| 10/15/2023 18:39:39 - INFO - app - STEP = 63 | |
| 10/15/2023 18:39:40 - INFO - app - STEP = 64 | |
| 10/15/2023 18:39:42 - INFO - app - STEP = 65 | |
| 10/15/2023 18:39:43 - INFO - app - STEP = 66 | |
| 10/15/2023 18:39:44 - INFO - app - STEP = 67 | |
| 10/15/2023 18:39:46 - INFO - app - STEP = 68 | |
| 10/15/2023 18:39:47 - INFO - app - STEP = 69 | |
| 10/15/2023 18:39:48 - INFO - app - STEP = 70 | |
| 10/15/2023 18:39:49 - INFO - app - STEP = 71 | |
| 10/15/2023 18:39:51 - INFO - app - STEP = 72 | |
| 10/15/2023 18:39:52 - INFO - app - STEP = 73 | |
| 10/15/2023 18:39:53 - INFO - app - STEP = 74 | |
| 10/15/2023 18:39:55 - INFO - app - STEP = 75 | |
| 10/15/2023 18:39:56 - INFO - app - STEP = 76 | |
| 10/15/2023 18:39:57 - INFO - app - STEP = 77 | |
| 10/15/2023 18:39:59 - INFO - app - STEP = 78 | |
| 10/15/2023 18:40:00 - INFO - app - STEP = 79 | |
| 10/15/2023 18:40:01 - INFO - app - STEP = 80 | |
| 10/15/2023 18:40:02 - INFO - app - STEP = 81 | |
| 10/15/2023 18:40:04 - INFO - app - STEP = 82 | |
| 10/15/2023 18:40:05 - INFO - app - STEP = 83 | |
| 10/15/2023 18:40:06 - INFO - app - STEP = 84 | |
| 10/15/2023 18:40:08 - INFO - app - STEP = 85 | |
| 10/15/2023 18:40:09 - INFO - app - STEP = 86 | |
| 10/15/2023 18:40:10 - INFO - app - STEP = 87 | |
| 10/15/2023 18:40:12 - INFO - app - STEP = 88 | |
| 10/15/2023 18:40:13 - INFO - app - STEP = 89 | |
| 10/15/2023 18:40:14 - INFO - app - STEP = 90 | |
| 10/15/2023 18:40:15 - INFO - app - STEP = 91 | |
| 10/15/2023 18:40:17 - INFO - app - STEP = 92 | |
| 10/15/2023 18:40:18 - INFO - app - STEP = 93 | |
| 10/15/2023 18:40:19 - INFO - app - STEP = 94 | |
| 10/15/2023 18:40:21 - INFO - app - STEP = 95 | |
| 10/15/2023 18:40:22 - INFO - app - STEP = 96 | |
| 10/15/2023 18:40:23 - INFO - app - STEP = 97 | |
| 10/15/2023 18:40:25 - INFO - app - STEP = 98 | |
| 10/15/2023 18:40:26 - INFO - app - STEP = 99 | |
| 10/15/2023 18:40:27 - INFO - app - STEP = 100 | |
| 10/15/2023 18:40:28 - INFO - app - STEP = 101 | |
| 10/15/2023 18:40:30 - INFO - app - STEP = 102 | |
| 10/15/2023 18:40:31 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:40:57 - INFO - app - STEP = 103 | |
| 10/15/2023 18:40:58 - INFO - app - STEP = 104 | |
| 10/15/2023 18:40:59 - INFO - app - STEP = 105 | |
| 10/15/2023 18:41:01 - INFO - app - STEP = 106 | |
| 10/15/2023 18:41:02 - INFO - app - STEP = 107 | |
| 10/15/2023 18:41:03 - INFO - app - STEP = 108 | |
| 10/15/2023 18:41:04 - INFO - app - STEP = 109 | |
| 10/15/2023 18:41:06 - INFO - app - STEP = 110 | |
| 10/15/2023 18:41:07 - INFO - app - STEP = 111 | |
| 10/15/2023 18:41:08 - INFO - app - STEP = 112 | |
| 10/15/2023 18:41:09 - INFO - app - STEP = 113 | |
| 10/15/2023 18:41:11 - INFO - app - STEP = 114 | |
| 10/15/2023 18:41:12 - INFO - app - STEP = 115 | |
| 10/15/2023 18:41:13 - INFO - app - STEP = 116 | |
| 10/15/2023 18:41:15 - INFO - app - STEP = 117 | |
| 10/15/2023 18:41:16 - INFO - app - STEP = 118 | |
| 10/15/2023 18:41:17 - INFO - app - STEP = 119 | |
| 10/15/2023 18:41:19 - INFO - app - STEP = 120 | |
| 10/15/2023 18:41:20 - INFO - app - STEP = 121 | |
| 10/15/2023 18:41:21 - INFO - app - STEP = 122 | |
| 10/15/2023 18:41:22 - INFO - app - STEP = 123 | |
| 10/15/2023 18:41:24 - INFO - app - STEP = 124 | |
| 10/15/2023 18:41:25 - INFO - app - STEP = 125 | |
| 10/15/2023 18:41:27 - INFO - app - STEP = 126 | |
| 10/15/2023 18:41:28 - INFO - app - STEP = 127 | |
| 10/15/2023 18:41:29 - INFO - app - STEP = 128 | |
| 10/15/2023 18:41:30 - INFO - app - STEP = 129 | |
| 10/15/2023 18:41:32 - INFO - app - STEP = 130 | |
| 10/15/2023 18:41:33 - INFO - app - STEP = 131 | |
| 10/15/2023 18:41:34 - INFO - app - STEP = 132 | |
| 10/15/2023 18:41:36 - INFO - app - STEP = 133 | |
| 10/15/2023 18:41:37 - INFO - app - STEP = 134 | |
| 10/15/2023 18:41:38 - INFO - app - STEP = 135 | |
| 10/15/2023 18:41:40 - INFO - app - STEP = 136 | |
| 10/15/2023 18:41:41 - INFO - app - STEP = 137 | |
| 10/15/2023 18:41:42 - INFO - app - STEP = 138 | |
| 10/15/2023 18:41:43 - INFO - app - STEP = 139 | |
| 10/15/2023 18:41:45 - INFO - app - STEP = 140 | |
| 10/15/2023 18:41:46 - INFO - app - STEP = 141 | |
| 10/15/2023 18:41:47 - INFO - app - STEP = 142 | |
| 10/15/2023 18:41:49 - INFO - app - STEP = 143 | |
| 10/15/2023 18:41:50 - INFO - app - STEP = 144 | |
| 10/15/2023 18:41:51 - INFO - app - STEP = 145 | |
| 10/15/2023 18:41:53 - INFO - app - STEP = 146 | |
| 10/15/2023 18:41:54 - INFO - app - STEP = 147 | |
| 10/15/2023 18:41:55 - INFO - app - STEP = 148 | |
| 10/15/2023 18:41:56 - INFO - app - STEP = 149 | |
| 10/15/2023 18:41:58 - INFO - app - STEP = 150 | |
| 10/15/2023 18:41:59 - INFO - app - STEP = 151 | |
| 10/15/2023 18:42:00 - INFO - app - STEP = 152 | |
| 10/15/2023 18:42:01 - INFO - app - STEP = 153 | |
| 10/15/2023 18:42:03 - INFO - app - STEP = 154 | |
| 10/15/2023 18:42:04 - INFO - app - STEP = 155 | |
| 10/15/2023 18:42:06 - INFO - app - STEP = 156 | |
| 10/15/2023 18:42:07 - INFO - app - STEP = 157 | |
| 10/15/2023 18:42:08 - INFO - app - STEP = 158 | |
| 10/15/2023 18:42:09 - INFO - app - STEP = 159 | |
| 10/15/2023 18:42:11 - INFO - app - STEP = 160 | |
| 10/15/2023 18:42:12 - INFO - app - STEP = 161 | |
| 10/15/2023 18:42:13 - INFO - app - STEP = 162 | |
| 10/15/2023 18:42:14 - INFO - app - STEP = 163 | |
| 10/15/2023 18:42:16 - INFO - app - STEP = 164 | |
| 10/15/2023 18:42:17 - INFO - app - STEP = 165 | |
| 10/15/2023 18:42:19 - INFO - app - STEP = 166 | |
| 10/15/2023 18:42:20 - INFO - app - STEP = 167 | |
| 10/15/2023 18:42:21 - INFO - app - STEP = 168 | |
| 10/15/2023 18:42:22 - INFO - app - STEP = 169 | |
| 10/15/2023 18:42:24 - INFO - app - STEP = 170 | |
| 10/15/2023 18:42:25 - INFO - app - STEP = 171 | |
| 10/15/2023 18:42:26 - INFO - app - STEP = 172 | |
| 10/15/2023 18:42:27 - INFO - app - STEP = 173 | |
| 10/15/2023 18:42:29 - INFO - app - STEP = 174 | |
| 10/15/2023 18:42:30 - INFO - app - STEP = 175 | |
| 10/15/2023 18:42:32 - INFO - app - STEP = 176 | |
| 10/15/2023 18:42:33 - INFO - app - STEP = 177 | |
| 10/15/2023 18:42:34 - INFO - app - STEP = 178 | |
| 10/15/2023 18:42:35 - INFO - app - STEP = 179 | |
| 10/15/2023 18:42:37 - INFO - app - STEP = 180 | |
| 10/15/2023 18:42:38 - INFO - app - STEP = 181 | |
| 10/15/2023 18:42:39 - INFO - app - STEP = 182 | |
| 10/15/2023 18:42:40 - INFO - app - STEP = 183 | |
| 10/15/2023 18:42:42 - INFO - app - STEP = 184 | |
| 10/15/2023 18:42:43 - INFO - app - STEP = 185 | |
| 10/15/2023 18:42:45 - INFO - app - STEP = 186 | |
| 10/15/2023 18:42:46 - INFO - app - STEP = 187 | |
| 10/15/2023 18:42:47 - INFO - app - STEP = 188 | |
| 10/15/2023 18:42:48 - INFO - app - STEP = 189 | |
| 10/15/2023 18:42:50 - INFO - app - STEP = 190 | |
| 10/15/2023 18:42:51 - INFO - app - STEP = 191 | |
| 10/15/2023 18:42:52 - INFO - app - STEP = 192 | |
| 10/15/2023 18:42:53 - INFO - app - STEP = 193 | |
| 10/15/2023 18:42:55 - INFO - app - STEP = 194 | |
| 10/15/2023 18:42:56 - INFO - app - STEP = 195 | |
| 10/15/2023 18:42:58 - INFO - app - STEP = 196 | |
| 10/15/2023 18:42:59 - INFO - app - STEP = 197 | |
| 10/15/2023 18:43:00 - INFO - app - STEP = 198 | |
| 10/15/2023 18:43:01 - INFO - app - STEP = 199 | |
| 10/15/2023 18:43:03 - INFO - app - STEP = 200 | |
| 10/15/2023 18:43:03 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-200 | |
| 10/15/2023 18:43:03 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-200/optimizer.bin | |
| 10/15/2023 18:43:03 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-200/scheduler.bin | |
| 10/15/2023 18:43:03 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-200/random_states_0.pkl | |
| 10/15/2023 18:43:03 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-200 | |
| 10/15/2023 18:43:04 - INFO - app - STEP = 201 | |
| 10/15/2023 18:43:05 - INFO - app - STEP = 202 | |
| 10/15/2023 18:43:06 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:43:32 - INFO - app - STEP = 203 | |
| 10/15/2023 18:43:34 - INFO - app - STEP = 204 | |
| 10/15/2023 18:43:35 - INFO - app - STEP = 205 | |
| 10/15/2023 18:43:36 - INFO - app - STEP = 206 | |
| 10/15/2023 18:43:38 - INFO - app - STEP = 207 | |
| 10/15/2023 18:43:39 - INFO - app - STEP = 208 | |
| 10/15/2023 18:43:40 - INFO - app - STEP = 209 | |
| 10/15/2023 18:43:42 - INFO - app - STEP = 210 | |
| 10/15/2023 18:43:43 - INFO - app - STEP = 211 | |
| 10/15/2023 18:43:44 - INFO - app - STEP = 212 | |
| 10/15/2023 18:43:45 - INFO - app - STEP = 213 | |
| 10/15/2023 18:43:47 - INFO - app - STEP = 214 | |
| 10/15/2023 18:43:48 - INFO - app - STEP = 215 | |
| 10/15/2023 18:43:49 - INFO - app - STEP = 216 | |
| 10/15/2023 18:43:51 - INFO - app - STEP = 217 | |
| 10/15/2023 18:43:52 - INFO - app - STEP = 218 | |
| 10/15/2023 18:43:53 - INFO - app - STEP = 219 | |
| 10/15/2023 18:43:55 - INFO - app - STEP = 220 | |
| 10/15/2023 18:43:56 - INFO - app - STEP = 221 | |
| 10/15/2023 18:43:57 - INFO - app - STEP = 222 | |
| 10/15/2023 18:43:59 - INFO - app - STEP = 223 | |
| 10/15/2023 18:44:00 - INFO - app - STEP = 224 | |
| 10/15/2023 18:44:01 - INFO - app - STEP = 225 | |
| 10/15/2023 18:44:03 - INFO - app - STEP = 226 | |
| 10/15/2023 18:44:04 - INFO - app - STEP = 227 | |
| 10/15/2023 18:44:05 - INFO - app - STEP = 228 | |
| 10/15/2023 18:44:06 - INFO - app - STEP = 229 | |
| 10/15/2023 18:44:08 - INFO - app - STEP = 230 | |
| 10/15/2023 18:44:09 - INFO - app - STEP = 231 | |
| 10/15/2023 18:44:10 - INFO - app - STEP = 232 | |
| 10/15/2023 18:44:12 - INFO - app - STEP = 233 | |
| 10/15/2023 18:44:13 - INFO - app - STEP = 234 | |
| 10/15/2023 18:44:14 - INFO - app - STEP = 235 | |
| 10/15/2023 18:44:16 - INFO - app - STEP = 236 | |
| 10/15/2023 18:44:17 - INFO - app - STEP = 237 | |
| 10/15/2023 18:44:18 - INFO - app - STEP = 238 | |
| 10/15/2023 18:44:19 - INFO - app - STEP = 239 | |
| 10/15/2023 18:44:21 - INFO - app - STEP = 240 | |
| 10/15/2023 18:44:22 - INFO - app - STEP = 241 | |
| 10/15/2023 18:44:24 - INFO - app - STEP = 242 | |
| 10/15/2023 18:44:25 - INFO - app - STEP = 243 | |
| 10/15/2023 18:44:26 - INFO - app - STEP = 244 | |
| 10/15/2023 18:44:27 - INFO - app - STEP = 245 | |
| 10/15/2023 18:44:29 - INFO - app - STEP = 246 | |
| 10/15/2023 18:44:30 - INFO - app - STEP = 247 | |
| 10/15/2023 18:44:31 - INFO - app - STEP = 248 | |
| 10/15/2023 18:44:33 - INFO - app - STEP = 249 | |
| 10/15/2023 18:44:34 - INFO - app - STEP = 250 | |
| 10/15/2023 18:44:35 - INFO - app - STEP = 251 | |
| 10/15/2023 18:44:37 - INFO - app - STEP = 252 | |
| 10/15/2023 18:44:38 - INFO - app - STEP = 253 | |
| 10/15/2023 18:44:39 - INFO - app - STEP = 254 | |
| 10/15/2023 18:44:40 - INFO - app - STEP = 255 | |
| 10/15/2023 18:44:42 - INFO - app - STEP = 256 | |
| 10/15/2023 18:44:43 - INFO - app - STEP = 257 | |
| 10/15/2023 18:44:44 - INFO - app - STEP = 258 | |
| 10/15/2023 18:44:46 - INFO - app - STEP = 259 | |
| 10/15/2023 18:44:47 - INFO - app - STEP = 260 | |
| 10/15/2023 18:44:48 - INFO - app - STEP = 261 | |
| 10/15/2023 18:44:50 - INFO - app - STEP = 262 | |
| 10/15/2023 18:44:51 - INFO - app - STEP = 263 | |
| 10/15/2023 18:44:52 - INFO - app - STEP = 264 | |
| 10/15/2023 18:44:53 - INFO - app - STEP = 265 | |
| 10/15/2023 18:44:55 - INFO - app - STEP = 266 | |
| 10/15/2023 18:44:56 - INFO - app - STEP = 267 | |
| 10/15/2023 18:44:57 - INFO - app - STEP = 268 | |
| 10/15/2023 18:44:59 - INFO - app - STEP = 269 | |
| 10/15/2023 18:45:00 - INFO - app - STEP = 270 | |
| 10/15/2023 18:45:01 - INFO - app - STEP = 271 | |
| 10/15/2023 18:45:03 - INFO - app - STEP = 272 | |
| 10/15/2023 18:45:04 - INFO - app - STEP = 273 | |
| 10/15/2023 18:45:05 - INFO - app - STEP = 274 | |
| 10/15/2023 18:45:06 - INFO - app - STEP = 275 | |
| 10/15/2023 18:45:08 - INFO - app - STEP = 276 | |
| 10/15/2023 18:45:09 - INFO - app - STEP = 277 | |
| 10/15/2023 18:45:10 - INFO - app - STEP = 278 | |
| 10/15/2023 18:45:12 - INFO - app - STEP = 279 | |
| 10/15/2023 18:45:13 - INFO - app - STEP = 280 | |
| 10/15/2023 18:45:14 - INFO - app - STEP = 281 | |
| 10/15/2023 18:45:16 - INFO - app - STEP = 282 | |
| 10/15/2023 18:45:17 - INFO - app - STEP = 283 | |
| 10/15/2023 18:45:18 - INFO - app - STEP = 284 | |
| 10/15/2023 18:45:19 - INFO - app - STEP = 285 | |
| 10/15/2023 18:45:21 - INFO - app - STEP = 286 | |
| 10/15/2023 18:45:22 - INFO - app - STEP = 287 | |
| 10/15/2023 18:45:23 - INFO - app - STEP = 288 | |
| 10/15/2023 18:45:25 - INFO - app - STEP = 289 | |
| 10/15/2023 18:45:26 - INFO - app - STEP = 290 | |
| 10/15/2023 18:45:27 - INFO - app - STEP = 291 | |
| 10/15/2023 18:45:29 - INFO - app - STEP = 292 | |
| 10/15/2023 18:45:30 - INFO - app - STEP = 293 | |
| 10/15/2023 18:45:31 - INFO - app - STEP = 294 | |
| 10/15/2023 18:45:33 - INFO - app - STEP = 295 | |
| 10/15/2023 18:45:34 - INFO - app - STEP = 296 | |
| 10/15/2023 18:45:35 - INFO - app - STEP = 297 | |
| 10/15/2023 18:45:37 - INFO - app - STEP = 298 | |
| 10/15/2023 18:45:38 - INFO - app - STEP = 299 | |
| 10/15/2023 18:45:39 - INFO - app - STEP = 300 | |
| 10/15/2023 18:45:40 - INFO - app - STEP = 301 | |
| 10/15/2023 18:45:42 - INFO - app - STEP = 302 | |
| 10/15/2023 18:45:42 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:46:08 - INFO - app - STEP = 303 | |
| 10/15/2023 18:46:10 - INFO - app - STEP = 304 | |
| 10/15/2023 18:46:11 - INFO - app - STEP = 305 | |
| 10/15/2023 18:46:12 - INFO - app - STEP = 306 | |
| 10/15/2023 18:46:14 - INFO - app - STEP = 307 | |
| 10/15/2023 18:46:15 - INFO - app - STEP = 308 | |
| 10/15/2023 18:46:16 - INFO - app - STEP = 309 | |
| 10/15/2023 18:46:18 - INFO - app - STEP = 310 | |
| 10/15/2023 18:46:19 - INFO - app - STEP = 311 | |
| 10/15/2023 18:46:20 - INFO - app - STEP = 312 | |
| 10/15/2023 18:46:21 - INFO - app - STEP = 313 | |
| 10/15/2023 18:46:23 - INFO - app - STEP = 314 | |
| 10/15/2023 18:46:24 - INFO - app - STEP = 315 | |
| 10/15/2023 18:46:25 - INFO - app - STEP = 316 | |
| 10/15/2023 18:46:27 - INFO - app - STEP = 317 | |
| 10/15/2023 18:46:28 - INFO - app - STEP = 318 | |
| 10/15/2023 18:46:29 - INFO - app - STEP = 319 | |
| 10/15/2023 18:46:31 - INFO - app - STEP = 320 | |
| 10/15/2023 18:46:32 - INFO - app - STEP = 321 | |
| 10/15/2023 18:46:33 - INFO - app - STEP = 322 | |
| 10/15/2023 18:46:35 - INFO - app - STEP = 323 | |
| 10/15/2023 18:46:36 - INFO - app - STEP = 324 | |
| 10/15/2023 18:46:37 - INFO - app - STEP = 325 | |
| 10/15/2023 18:46:39 - INFO - app - STEP = 326 | |
| 10/15/2023 18:46:40 - INFO - app - STEP = 327 | |
| 10/15/2023 18:46:41 - INFO - app - STEP = 328 | |
| 10/15/2023 18:46:42 - INFO - app - STEP = 329 | |
| 10/15/2023 18:46:44 - INFO - app - STEP = 330 | |
| 10/15/2023 18:46:45 - INFO - app - STEP = 331 | |
| 10/15/2023 18:46:47 - INFO - app - STEP = 332 | |
| 10/15/2023 18:46:48 - INFO - app - STEP = 333 | |
| 10/15/2023 18:46:49 - INFO - app - STEP = 334 | |
| 10/15/2023 18:46:50 - INFO - app - STEP = 335 | |
| 10/15/2023 18:46:52 - INFO - app - STEP = 336 | |
| 10/15/2023 18:46:53 - INFO - app - STEP = 337 | |
| 10/15/2023 18:46:54 - INFO - app - STEP = 338 | |
| 10/15/2023 18:46:56 - INFO - app - STEP = 339 | |
| 10/15/2023 18:46:57 - INFO - app - STEP = 340 | |
| 10/15/2023 18:46:58 - INFO - app - STEP = 341 | |
| 10/15/2023 18:47:00 - INFO - app - STEP = 342 | |
| 10/15/2023 18:47:01 - INFO - app - STEP = 343 | |
| 10/15/2023 18:47:02 - INFO - app - STEP = 344 | |
| 10/15/2023 18:47:04 - INFO - app - STEP = 345 | |
| 10/15/2023 18:47:05 - INFO - app - STEP = 346 | |
| 10/15/2023 18:47:06 - INFO - app - STEP = 347 | |
| 10/15/2023 18:47:08 - INFO - app - STEP = 348 | |
| 10/15/2023 18:47:09 - INFO - app - STEP = 349 | |
| 10/15/2023 18:47:10 - INFO - app - STEP = 350 | |
| 10/15/2023 18:47:11 - INFO - app - STEP = 351 | |
| 10/15/2023 18:47:13 - INFO - app - STEP = 352 | |
| 10/15/2023 18:47:14 - INFO - app - STEP = 353 | |
| 10/15/2023 18:47:15 - INFO - app - STEP = 354 | |
| 10/15/2023 18:47:17 - INFO - app - STEP = 355 | |
| 10/15/2023 18:47:18 - INFO - app - STEP = 356 | |
| 10/15/2023 18:47:19 - INFO - app - STEP = 357 | |
| 10/15/2023 18:47:21 - INFO - app - STEP = 358 | |
| 10/15/2023 18:47:22 - INFO - app - STEP = 359 | |
| 10/15/2023 18:47:23 - INFO - app - STEP = 360 | |
| 10/15/2023 18:47:25 - INFO - app - STEP = 361 | |
| 10/15/2023 18:47:26 - INFO - app - STEP = 362 | |
| 10/15/2023 18:47:27 - INFO - app - STEP = 363 | |
| 10/15/2023 18:47:29 - INFO - app - STEP = 364 | |
| 10/15/2023 18:47:30 - INFO - app - STEP = 365 | |
| 10/15/2023 18:47:31 - INFO - app - STEP = 366 | |
| 10/15/2023 18:47:32 - INFO - app - STEP = 367 | |
| 10/15/2023 18:47:34 - INFO - app - STEP = 368 | |
| 10/15/2023 18:47:35 - INFO - app - STEP = 369 | |
| 10/15/2023 18:47:37 - INFO - app - STEP = 370 | |
| 10/15/2023 18:47:38 - INFO - app - STEP = 371 | |
| 10/15/2023 18:47:39 - INFO - app - STEP = 372 | |
| 10/15/2023 18:47:40 - INFO - app - STEP = 373 | |
| 10/15/2023 18:47:42 - INFO - app - STEP = 374 | |
| 10/15/2023 18:47:43 - INFO - app - STEP = 375 | |
| 10/15/2023 18:47:44 - INFO - app - STEP = 376 | |
| 10/15/2023 18:47:46 - INFO - app - STEP = 377 | |
| 10/15/2023 18:47:47 - INFO - app - STEP = 378 | |
| 10/15/2023 18:47:48 - INFO - app - STEP = 379 | |
| 10/15/2023 18:47:50 - INFO - app - STEP = 380 | |
| 10/15/2023 18:47:51 - INFO - app - STEP = 381 | |
| 10/15/2023 18:47:52 - INFO - app - STEP = 382 | |
| 10/15/2023 18:47:53 - INFO - app - STEP = 383 | |
| 10/15/2023 18:47:55 - INFO - app - STEP = 384 | |
| 10/15/2023 18:47:56 - INFO - app - STEP = 385 | |
| 10/15/2023 18:47:57 - INFO - app - STEP = 386 | |
| 10/15/2023 18:47:59 - INFO - app - STEP = 387 | |
| 10/15/2023 18:48:00 - INFO - app - STEP = 388 | |
| 10/15/2023 18:48:01 - INFO - app - STEP = 389 | |
| 10/15/2023 18:48:03 - INFO - app - STEP = 390 | |
| 10/15/2023 18:48:04 - INFO - app - STEP = 391 | |
| 10/15/2023 18:48:05 - INFO - app - STEP = 392 | |
| 10/15/2023 18:48:07 - INFO - app - STEP = 393 | |
| 10/15/2023 18:48:08 - INFO - app - STEP = 394 | |
| 10/15/2023 18:48:09 - INFO - app - STEP = 395 | |
| 10/15/2023 18:48:11 - INFO - app - STEP = 396 | |
| 10/15/2023 18:48:12 - INFO - app - STEP = 397 | |
| 10/15/2023 18:48:13 - INFO - app - STEP = 398 | |
| 10/15/2023 18:48:15 - INFO - app - STEP = 399 | |
| 10/15/2023 18:48:16 - INFO - app - STEP = 400 | |
| 10/15/2023 18:48:17 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-400 | |
| 10/15/2023 18:48:17 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-400/optimizer.bin | |
| 10/15/2023 18:48:17 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-400/scheduler.bin | |
| 10/15/2023 18:48:17 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-400/random_states_0.pkl | |
| 10/15/2023 18:48:17 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-400 | |
| 10/15/2023 18:48:17 - INFO - app - STEP = 401 | |
| 10/15/2023 18:48:19 - INFO - app - STEP = 402 | |
| 10/15/2023 18:48:19 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:48:45 - INFO - app - STEP = 403 | |
| 10/15/2023 18:48:47 - INFO - app - STEP = 404 | |
| 10/15/2023 18:48:48 - INFO - app - STEP = 405 | |
| 10/15/2023 18:48:49 - INFO - app - STEP = 406 | |
| 10/15/2023 18:48:50 - INFO - app - STEP = 407 | |
| 10/15/2023 18:48:52 - INFO - app - STEP = 408 | |
| 10/15/2023 18:48:53 - INFO - app - STEP = 409 | |
| 10/15/2023 18:48:54 - INFO - app - STEP = 410 | |
| 10/15/2023 18:48:56 - INFO - app - STEP = 411 | |
| 10/15/2023 18:48:57 - INFO - app - STEP = 412 | |
| 10/15/2023 18:48:58 - INFO - app - STEP = 413 | |
| 10/15/2023 18:49:00 - INFO - app - STEP = 414 | |
| 10/15/2023 18:49:01 - INFO - app - STEP = 415 | |
| 10/15/2023 18:49:02 - INFO - app - STEP = 416 | |
| 10/15/2023 18:49:04 - INFO - app - STEP = 417 | |
| 10/15/2023 18:49:05 - INFO - app - STEP = 418 | |
| 10/15/2023 18:49:06 - INFO - app - STEP = 419 | |
| 10/15/2023 18:49:08 - INFO - app - STEP = 420 | |
| 10/15/2023 18:49:09 - INFO - app - STEP = 421 | |
| 10/15/2023 18:49:10 - INFO - app - STEP = 422 | |
| 10/15/2023 18:49:12 - INFO - app - STEP = 423 | |
| 10/15/2023 18:49:13 - INFO - app - STEP = 424 | |
| 10/15/2023 18:49:14 - INFO - app - STEP = 425 | |
| 10/15/2023 18:49:16 - INFO - app - STEP = 426 | |
| 10/15/2023 18:49:17 - INFO - app - STEP = 427 | |
| 10/15/2023 18:49:18 - INFO - app - STEP = 428 | |
| 10/15/2023 18:49:19 - INFO - app - STEP = 429 | |
| 10/15/2023 18:49:21 - INFO - app - STEP = 430 | |
| 10/15/2023 18:49:22 - INFO - app - STEP = 431 | |
| 10/15/2023 18:49:23 - INFO - app - STEP = 432 | |
| 10/15/2023 18:49:25 - INFO - app - STEP = 433 | |
| 10/15/2023 18:49:26 - INFO - app - STEP = 434 | |
| 10/15/2023 18:49:27 - INFO - app - STEP = 435 | |
| 10/15/2023 18:49:29 - INFO - app - STEP = 436 | |
| 10/15/2023 18:49:30 - INFO - app - STEP = 437 | |
| 10/15/2023 18:49:31 - INFO - app - STEP = 438 | |
| 10/15/2023 18:49:33 - INFO - app - STEP = 439 | |
| 10/15/2023 18:49:34 - INFO - app - STEP = 440 | |
| 10/15/2023 18:49:35 - INFO - app - STEP = 441 | |
| 10/15/2023 18:49:37 - INFO - app - STEP = 442 | |
| 10/15/2023 18:49:38 - INFO - app - STEP = 443 | |
| 10/15/2023 18:49:39 - INFO - app - STEP = 444 | |
| 10/15/2023 18:49:40 - INFO - app - STEP = 445 | |
| 10/15/2023 18:49:42 - INFO - app - STEP = 446 | |
| 10/15/2023 18:49:43 - INFO - app - STEP = 447 | |
| 10/15/2023 18:49:44 - INFO - app - STEP = 448 | |
| 10/15/2023 18:49:46 - INFO - app - STEP = 449 | |
| 10/15/2023 18:49:47 - INFO - app - STEP = 450 | |
| 10/15/2023 18:49:48 - INFO - app - STEP = 451 | |
| 10/15/2023 18:49:50 - INFO - app - STEP = 452 | |
| 10/15/2023 18:49:51 - INFO - app - STEP = 453 | |
| 10/15/2023 18:49:52 - INFO - app - STEP = 454 | |
| 10/15/2023 18:49:54 - INFO - app - STEP = 455 | |
| 10/15/2023 18:49:55 - INFO - app - STEP = 456 | |
| 10/15/2023 18:49:56 - INFO - app - STEP = 457 | |
| 10/15/2023 18:49:58 - INFO - app - STEP = 458 | |
| 10/15/2023 18:49:59 - INFO - app - STEP = 459 | |
| 10/15/2023 18:50:00 - INFO - app - STEP = 460 | |
| 10/15/2023 18:50:01 - INFO - app - STEP = 461 | |
| 10/15/2023 18:50:03 - INFO - app - STEP = 462 | |
| 10/15/2023 18:50:04 - INFO - app - STEP = 463 | |
| 10/15/2023 18:50:05 - INFO - app - STEP = 464 | |
| 10/15/2023 18:50:07 - INFO - app - STEP = 465 | |
| 10/15/2023 18:50:08 - INFO - app - STEP = 466 | |
| 10/15/2023 18:50:09 - INFO - app - STEP = 467 | |
| 10/15/2023 18:50:11 - INFO - app - STEP = 468 | |
| 10/15/2023 18:50:12 - INFO - app - STEP = 469 | |
| 10/15/2023 18:50:13 - INFO - app - STEP = 470 | |
| 10/15/2023 18:50:15 - INFO - app - STEP = 471 | |
| 10/15/2023 18:50:16 - INFO - app - STEP = 472 | |
| 10/15/2023 18:50:17 - INFO - app - STEP = 473 | |
| 10/15/2023 18:50:19 - INFO - app - STEP = 474 | |
| 10/15/2023 18:50:20 - INFO - app - STEP = 475 | |
| 10/15/2023 18:50:21 - INFO - app - STEP = 476 | |
| 10/15/2023 18:50:22 - INFO - app - STEP = 477 | |
| 10/15/2023 18:50:24 - INFO - app - STEP = 478 | |
| 10/15/2023 18:50:25 - INFO - app - STEP = 479 | |
| 10/15/2023 18:50:26 - INFO - app - STEP = 480 | |
| 10/15/2023 18:50:27 - INFO - app - STEP = 481 | |
| 10/15/2023 18:50:29 - INFO - app - STEP = 482 | |
| 10/15/2023 18:50:30 - INFO - app - STEP = 483 | |
| 10/15/2023 18:50:32 - INFO - app - STEP = 484 | |
| 10/15/2023 18:50:33 - INFO - app - STEP = 485 | |
| 10/15/2023 18:50:34 - INFO - app - STEP = 486 | |
| 10/15/2023 18:50:35 - INFO - app - STEP = 487 | |
| 10/15/2023 18:50:37 - INFO - app - STEP = 488 | |
| 10/15/2023 18:50:38 - INFO - app - STEP = 489 | |
| 10/15/2023 18:50:39 - INFO - app - STEP = 490 | |
| 10/15/2023 18:50:41 - INFO - app - STEP = 491 | |
| 10/15/2023 18:50:42 - INFO - app - STEP = 492 | |
| 10/15/2023 18:50:43 - INFO - app - STEP = 493 | |
| 10/15/2023 18:50:45 - INFO - app - STEP = 494 | |
| 10/15/2023 18:50:46 - INFO - app - STEP = 495 | |
| 10/15/2023 18:50:47 - INFO - app - STEP = 496 | |
| 10/15/2023 18:50:49 - INFO - app - STEP = 497 | |
| 10/15/2023 18:50:50 - INFO - app - STEP = 498 | |
| 10/15/2023 18:50:51 - INFO - app - STEP = 499 | |
| 10/15/2023 18:50:53 - INFO - app - STEP = 500 | |
| 10/15/2023 18:50:54 - INFO - app - STEP = 501 | |
| 10/15/2023 18:50:55 - INFO - app - STEP = 502 | |
| 10/15/2023 18:50:56 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:51:22 - INFO - app - STEP = 503 | |
| 10/15/2023 18:51:23 - INFO - app - STEP = 504 | |
| 10/15/2023 18:51:24 - INFO - app - STEP = 505 | |
| 10/15/2023 18:51:26 - INFO - app - STEP = 506 | |
| 10/15/2023 18:51:27 - INFO - app - STEP = 507 | |
| 10/15/2023 18:51:28 - INFO - app - STEP = 508 | |
| 10/15/2023 18:51:30 - INFO - app - STEP = 509 | |
| 10/15/2023 18:51:31 - INFO - app - STEP = 510 | |
| 10/15/2023 18:51:32 - INFO - app - STEP = 511 | |
| 10/15/2023 18:51:34 - INFO - app - STEP = 512 | |
| 10/15/2023 18:51:35 - INFO - app - STEP = 513 | |
| 10/15/2023 18:51:36 - INFO - app - STEP = 514 | |
| 10/15/2023 18:51:38 - INFO - app - STEP = 515 | |
| 10/15/2023 18:51:39 - INFO - app - STEP = 516 | |
| 10/15/2023 18:51:40 - INFO - app - STEP = 517 | |
| 10/15/2023 18:51:42 - INFO - app - STEP = 518 | |
| 10/15/2023 18:51:43 - INFO - app - STEP = 519 | |
| 10/15/2023 18:51:44 - INFO - app - STEP = 520 | |
| 10/15/2023 18:51:45 - INFO - app - STEP = 521 | |
| 10/15/2023 18:51:47 - INFO - app - STEP = 522 | |
| 10/15/2023 18:51:48 - INFO - app - STEP = 523 | |
| 10/15/2023 18:51:50 - INFO - app - STEP = 524 | |
| 10/15/2023 18:51:51 - INFO - app - STEP = 525 | |
| 10/15/2023 18:51:52 - INFO - app - STEP = 526 | |
| 10/15/2023 18:51:53 - INFO - app - STEP = 527 | |
| 10/15/2023 18:51:55 - INFO - app - STEP = 528 | |
| 10/15/2023 18:51:56 - INFO - app - STEP = 529 | |
| 10/15/2023 18:51:58 - INFO - app - STEP = 530 | |
| 10/15/2023 18:51:59 - INFO - app - STEP = 531 | |
| 10/15/2023 18:52:00 - INFO - app - STEP = 532 | |
| 10/15/2023 18:52:01 - INFO - app - STEP = 533 | |
| 10/15/2023 18:52:03 - INFO - app - STEP = 534 | |
| 10/15/2023 18:52:04 - INFO - app - STEP = 535 | |
| 10/15/2023 18:52:05 - INFO - app - STEP = 536 | |
| 10/15/2023 18:52:07 - INFO - app - STEP = 537 | |
| 10/15/2023 18:52:08 - INFO - app - STEP = 538 | |
| 10/15/2023 18:52:09 - INFO - app - STEP = 539 | |
| 10/15/2023 18:52:11 - INFO - app - STEP = 540 | |
| 10/15/2023 18:52:12 - INFO - app - STEP = 541 | |
| 10/15/2023 18:52:13 - INFO - app - STEP = 542 | |
| 10/15/2023 18:52:14 - INFO - app - STEP = 543 | |
| 10/15/2023 18:52:16 - INFO - app - STEP = 544 | |
| 10/15/2023 18:52:17 - INFO - app - STEP = 545 | |
| 10/15/2023 18:52:18 - INFO - app - STEP = 546 | |
| 10/15/2023 18:52:20 - INFO - app - STEP = 547 | |
| 10/15/2023 18:52:21 - INFO - app - STEP = 548 | |
| 10/15/2023 18:52:22 - INFO - app - STEP = 549 | |
| 10/15/2023 18:52:24 - INFO - app - STEP = 550 | |
| 10/15/2023 18:52:25 - INFO - app - STEP = 551 | |
| 10/15/2023 18:52:26 - INFO - app - STEP = 552 | |
| 10/15/2023 18:52:27 - INFO - app - STEP = 553 | |
| 10/15/2023 18:52:29 - INFO - app - STEP = 554 | |
| 10/15/2023 18:52:30 - INFO - app - STEP = 555 | |
| 10/15/2023 18:52:31 - INFO - app - STEP = 556 | |
| 10/15/2023 18:52:33 - INFO - app - STEP = 557 | |
| 10/15/2023 18:52:34 - INFO - app - STEP = 558 | |
| 10/15/2023 18:52:35 - INFO - app - STEP = 559 | |
| 10/15/2023 18:52:37 - INFO - app - STEP = 560 | |
| 10/15/2023 18:52:38 - INFO - app - STEP = 561 | |
| 10/15/2023 18:52:39 - INFO - app - STEP = 562 | |
| 10/15/2023 18:52:40 - INFO - app - STEP = 563 | |
| 10/15/2023 18:52:42 - INFO - app - STEP = 564 | |
| 10/15/2023 18:52:43 - INFO - app - STEP = 565 | |
| 10/15/2023 18:52:44 - INFO - app - STEP = 566 | |
| 10/15/2023 18:52:46 - INFO - app - STEP = 567 | |
| 10/15/2023 18:52:47 - INFO - app - STEP = 568 | |
| 10/15/2023 18:52:48 - INFO - app - STEP = 569 | |
| 10/15/2023 18:52:50 - INFO - app - STEP = 570 | |
| 10/15/2023 18:52:51 - INFO - app - STEP = 571 | |
| 10/15/2023 18:52:52 - INFO - app - STEP = 572 | |
| 10/15/2023 18:52:53 - INFO - app - STEP = 573 | |
| 10/15/2023 18:52:55 - INFO - app - STEP = 574 | |
| 10/15/2023 18:52:56 - INFO - app - STEP = 575 | |
| 10/15/2023 18:52:57 - INFO - app - STEP = 576 | |
| 10/15/2023 18:52:59 - INFO - app - STEP = 577 | |
| 10/15/2023 18:53:00 - INFO - app - STEP = 578 | |
| 10/15/2023 18:53:01 - INFO - app - STEP = 579 | |
| 10/15/2023 18:53:03 - INFO - app - STEP = 580 | |
| 10/15/2023 18:53:04 - INFO - app - STEP = 581 | |
| 10/15/2023 18:53:05 - INFO - app - STEP = 582 | |
| 10/15/2023 18:53:07 - INFO - app - STEP = 583 | |
| 10/15/2023 18:53:08 - INFO - app - STEP = 584 | |
| 10/15/2023 18:53:09 - INFO - app - STEP = 585 | |
| 10/15/2023 18:53:11 - INFO - app - STEP = 586 | |
| 10/15/2023 18:53:12 - INFO - app - STEP = 587 | |
| 10/15/2023 18:53:13 - INFO - app - STEP = 588 | |
| 10/15/2023 18:53:14 - INFO - app - STEP = 589 | |
| 10/15/2023 18:53:16 - INFO - app - STEP = 590 | |
| 10/15/2023 18:53:17 - INFO - app - STEP = 591 | |
| 10/15/2023 18:53:18 - INFO - app - STEP = 592 | |
| 10/15/2023 18:53:20 - INFO - app - STEP = 593 | |
| 10/15/2023 18:53:21 - INFO - app - STEP = 594 | |
| 10/15/2023 18:53:22 - INFO - app - STEP = 595 | |
| 10/15/2023 18:53:24 - INFO - app - STEP = 596 | |
| 10/15/2023 18:53:25 - INFO - app - STEP = 597 | |
| 10/15/2023 18:53:26 - INFO - app - STEP = 598 | |
| 10/15/2023 18:53:27 - INFO - app - STEP = 599 | |
| 10/15/2023 18:53:29 - INFO - app - STEP = 600 | |
| 10/15/2023 18:53:30 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-600 | |
| 10/15/2023 18:53:30 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-600/optimizer.bin | |
| 10/15/2023 18:53:30 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-600/scheduler.bin | |
| 10/15/2023 18:53:30 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-600/random_states_0.pkl | |
| 10/15/2023 18:53:30 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-600 | |
| 10/15/2023 18:53:30 - INFO - app - STEP = 601 | |
| 10/15/2023 18:53:31 - INFO - app - STEP = 602 | |
| 10/15/2023 18:53:32 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:53:58 - INFO - app - STEP = 603 | |
| 10/15/2023 18:53:59 - INFO - app - STEP = 604 | |
| 10/15/2023 18:54:01 - INFO - app - STEP = 605 | |
| 10/15/2023 18:54:02 - INFO - app - STEP = 606 | |
| 10/15/2023 18:54:03 - INFO - app - STEP = 607 | |
| 10/15/2023 18:54:05 - INFO - app - STEP = 608 | |
| 10/15/2023 18:54:06 - INFO - app - STEP = 609 | |
| 10/15/2023 18:54:07 - INFO - app - STEP = 610 | |
| 10/15/2023 18:54:08 - INFO - app - STEP = 611 | |
| 10/15/2023 18:54:10 - INFO - app - STEP = 612 | |
| 10/15/2023 18:54:11 - INFO - app - STEP = 613 | |
| 10/15/2023 18:54:13 - INFO - app - STEP = 614 | |
| 10/15/2023 18:54:14 - INFO - app - STEP = 615 | |
| 10/15/2023 18:54:15 - INFO - app - STEP = 616 | |
| 10/15/2023 18:54:16 - INFO - app - STEP = 617 | |
| 10/15/2023 18:54:18 - INFO - app - STEP = 618 | |
| 10/15/2023 18:54:19 - INFO - app - STEP = 619 | |
| 10/15/2023 18:54:20 - INFO - app - STEP = 620 | |
| 10/15/2023 18:54:22 - INFO - app - STEP = 621 | |
| 10/15/2023 18:54:23 - INFO - app - STEP = 622 | |
| 10/15/2023 18:54:24 - INFO - app - STEP = 623 | |
| 10/15/2023 18:54:26 - INFO - app - STEP = 624 | |
| 10/15/2023 18:54:27 - INFO - app - STEP = 625 | |
| 10/15/2023 18:54:28 - INFO - app - STEP = 626 | |
| 10/15/2023 18:54:29 - INFO - app - STEP = 627 | |
| 10/15/2023 18:54:31 - INFO - app - STEP = 628 | |
| 10/15/2023 18:54:32 - INFO - app - STEP = 629 | |
| 10/15/2023 18:54:34 - INFO - app - STEP = 630 | |
| 10/15/2023 18:54:35 - INFO - app - STEP = 631 | |
| 10/15/2023 18:54:36 - INFO - app - STEP = 632 | |
| 10/15/2023 18:54:37 - INFO - app - STEP = 633 | |
| 10/15/2023 18:54:39 - INFO - app - STEP = 634 | |
| 10/15/2023 18:54:40 - INFO - app - STEP = 635 | |
| 10/15/2023 18:54:41 - INFO - app - STEP = 636 | |
| 10/15/2023 18:54:43 - INFO - app - STEP = 637 | |
| 10/15/2023 18:54:44 - INFO - app - STEP = 638 | |
| 10/15/2023 18:54:45 - INFO - app - STEP = 639 | |
| 10/15/2023 18:54:47 - INFO - app - STEP = 640 | |
| 10/15/2023 18:54:48 - INFO - app - STEP = 641 | |
| 10/15/2023 18:54:49 - INFO - app - STEP = 642 | |
| 10/15/2023 18:54:50 - INFO - app - STEP = 643 | |
| 10/15/2023 18:54:52 - INFO - app - STEP = 644 | |
| 10/15/2023 18:54:53 - INFO - app - STEP = 645 | |
| 10/15/2023 18:54:54 - INFO - app - STEP = 646 | |
| 10/15/2023 18:54:56 - INFO - app - STEP = 647 | |
| 10/15/2023 18:54:57 - INFO - app - STEP = 648 | |
| 10/15/2023 18:54:58 - INFO - app - STEP = 649 | |
| 10/15/2023 18:55:00 - INFO - app - STEP = 650 | |
| 10/15/2023 18:55:01 - INFO - app - STEP = 651 | |
| 10/15/2023 18:55:02 - INFO - app - STEP = 652 | |
| 10/15/2023 18:55:04 - INFO - app - STEP = 653 | |
| 10/15/2023 18:55:05 - INFO - app - STEP = 654 | |
| 10/15/2023 18:55:06 - INFO - app - STEP = 655 | |
| 10/15/2023 18:55:08 - INFO - app - STEP = 656 | |
| 10/15/2023 18:55:09 - INFO - app - STEP = 657 | |
| 10/15/2023 18:55:10 - INFO - app - STEP = 658 | |
| 10/15/2023 18:55:11 - INFO - app - STEP = 659 | |
| 10/15/2023 18:55:13 - INFO - app - STEP = 660 | |
| 10/15/2023 18:55:14 - INFO - app - STEP = 661 | |
| 10/15/2023 18:55:16 - INFO - app - STEP = 662 | |
| 10/15/2023 18:55:17 - INFO - app - STEP = 663 | |
| 10/15/2023 18:55:18 - INFO - app - STEP = 664 | |
| 10/15/2023 18:55:19 - INFO - app - STEP = 665 | |
| 10/15/2023 18:55:21 - INFO - app - STEP = 666 | |
| 10/15/2023 18:55:22 - INFO - app - STEP = 667 | |
| 10/15/2023 18:55:23 - INFO - app - STEP = 668 | |
| 10/15/2023 18:55:25 - INFO - app - STEP = 669 | |
| 10/15/2023 18:55:26 - INFO - app - STEP = 670 | |
| 10/15/2023 18:55:27 - INFO - app - STEP = 671 | |
| 10/15/2023 18:55:29 - INFO - app - STEP = 672 | |
| 10/15/2023 18:55:30 - INFO - app - STEP = 673 | |
| 10/15/2023 18:55:31 - INFO - app - STEP = 674 | |
| 10/15/2023 18:55:32 - INFO - app - STEP = 675 | |
| 10/15/2023 18:55:34 - INFO - app - STEP = 676 | |
| 10/15/2023 18:55:35 - INFO - app - STEP = 677 | |
| 10/15/2023 18:55:37 - INFO - app - STEP = 678 | |
| 10/15/2023 18:55:38 - INFO - app - STEP = 679 | |
| 10/15/2023 18:55:39 - INFO - app - STEP = 680 | |
| 10/15/2023 18:55:40 - INFO - app - STEP = 681 | |
| 10/15/2023 18:55:42 - INFO - app - STEP = 682 | |
| 10/15/2023 18:55:43 - INFO - app - STEP = 683 | |
| 10/15/2023 18:55:44 - INFO - app - STEP = 684 | |
| 10/15/2023 18:55:46 - INFO - app - STEP = 685 | |
| 10/15/2023 18:55:47 - INFO - app - STEP = 686 | |
| 10/15/2023 18:55:48 - INFO - app - STEP = 687 | |
| 10/15/2023 18:55:50 - INFO - app - STEP = 688 | |
| 10/15/2023 18:55:51 - INFO - app - STEP = 689 | |
| 10/15/2023 18:55:52 - INFO - app - STEP = 690 | |
| 10/15/2023 18:55:53 - INFO - app - STEP = 691 | |
| 10/15/2023 18:55:55 - INFO - app - STEP = 692 | |
| 10/15/2023 18:55:56 - INFO - app - STEP = 693 | |
| 10/15/2023 18:55:57 - INFO - app - STEP = 694 | |
| 10/15/2023 18:55:59 - INFO - app - STEP = 695 | |
| 10/15/2023 18:56:00 - INFO - app - STEP = 696 | |
| 10/15/2023 18:56:01 - INFO - app - STEP = 697 | |
| 10/15/2023 18:56:03 - INFO - app - STEP = 698 | |
| 10/15/2023 18:56:04 - INFO - app - STEP = 699 | |
| 10/15/2023 18:56:05 - INFO - app - STEP = 700 | |
| 10/15/2023 18:56:06 - INFO - app - STEP = 701 | |
| 10/15/2023 18:56:08 - INFO - app - STEP = 702 | |
| 10/15/2023 18:56:09 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:56:34 - INFO - app - STEP = 703 | |
| 10/15/2023 18:56:36 - INFO - app - STEP = 704 | |
| 10/15/2023 18:56:37 - INFO - app - STEP = 705 | |
| 10/15/2023 18:56:38 - INFO - app - STEP = 706 | |
| 10/15/2023 18:56:40 - INFO - app - STEP = 707 | |
| 10/15/2023 18:56:41 - INFO - app - STEP = 708 | |
| 10/15/2023 18:56:42 - INFO - app - STEP = 709 | |
| 10/15/2023 18:56:44 - INFO - app - STEP = 710 | |
| 10/15/2023 18:56:45 - INFO - app - STEP = 711 | |
| 10/15/2023 18:56:46 - INFO - app - STEP = 712 | |
| 10/15/2023 18:56:47 - INFO - app - STEP = 713 | |
| 10/15/2023 18:56:49 - INFO - app - STEP = 714 | |
| 10/15/2023 18:56:50 - INFO - app - STEP = 715 | |
| 10/15/2023 18:56:51 - INFO - app - STEP = 716 | |
| 10/15/2023 18:56:52 - INFO - app - STEP = 717 | |
| 10/15/2023 18:56:54 - INFO - app - STEP = 718 | |
| 10/15/2023 18:56:55 - INFO - app - STEP = 719 | |
| 10/15/2023 18:56:56 - INFO - app - STEP = 720 | |
| 10/15/2023 18:56:58 - INFO - app - STEP = 721 | |
| 10/15/2023 18:56:59 - INFO - app - STEP = 722 | |
| 10/15/2023 18:57:00 - INFO - app - STEP = 723 | |
| 10/15/2023 18:57:02 - INFO - app - STEP = 724 | |
| 10/15/2023 18:57:03 - INFO - app - STEP = 725 | |
| 10/15/2023 18:57:04 - INFO - app - STEP = 726 | |
| 10/15/2023 18:57:05 - INFO - app - STEP = 727 | |
| 10/15/2023 18:57:07 - INFO - app - STEP = 728 | |
| 10/15/2023 18:57:08 - INFO - app - STEP = 729 | |
| 10/15/2023 18:57:09 - INFO - app - STEP = 730 | |
| 10/15/2023 18:57:11 - INFO - app - STEP = 731 | |
| 10/15/2023 18:57:12 - INFO - app - STEP = 732 | |
| 10/15/2023 18:57:13 - INFO - app - STEP = 733 | |
| 10/15/2023 18:57:15 - INFO - app - STEP = 734 | |
| 10/15/2023 18:57:16 - INFO - app - STEP = 735 | |
| 10/15/2023 18:57:17 - INFO - app - STEP = 736 | |
| 10/15/2023 18:57:18 - INFO - app - STEP = 737 | |
| 10/15/2023 18:57:20 - INFO - app - STEP = 738 | |
| 10/15/2023 18:57:21 - INFO - app - STEP = 739 | |
| 10/15/2023 18:57:22 - INFO - app - STEP = 740 | |
| 10/15/2023 18:57:23 - INFO - app - STEP = 741 | |
| 10/15/2023 18:57:25 - INFO - app - STEP = 742 | |
| 10/15/2023 18:57:26 - INFO - app - STEP = 743 | |
| 10/15/2023 18:57:27 - INFO - app - STEP = 744 | |
| 10/15/2023 18:57:29 - INFO - app - STEP = 745 | |
| 10/15/2023 18:57:30 - INFO - app - STEP = 746 | |
| 10/15/2023 18:57:31 - INFO - app - STEP = 747 | |
| 10/15/2023 18:57:33 - INFO - app - STEP = 748 | |
| 10/15/2023 18:57:34 - INFO - app - STEP = 749 | |
| 10/15/2023 18:57:35 - INFO - app - STEP = 750 | |
| 10/15/2023 18:57:36 - INFO - app - STEP = 751 | |
| 10/15/2023 18:57:38 - INFO - app - STEP = 752 | |
| 10/15/2023 18:57:39 - INFO - app - STEP = 753 | |
| 10/15/2023 18:57:40 - INFO - app - STEP = 754 | |
| 10/15/2023 18:57:41 - INFO - app - STEP = 755 | |
| 10/15/2023 18:57:43 - INFO - app - STEP = 756 | |
| 10/15/2023 18:57:44 - INFO - app - STEP = 757 | |
| 10/15/2023 18:57:45 - INFO - app - STEP = 758 | |
| 10/15/2023 18:57:47 - INFO - app - STEP = 759 | |
| 10/15/2023 18:57:48 - INFO - app - STEP = 760 | |
| 10/15/2023 18:57:49 - INFO - app - STEP = 761 | |
| 10/15/2023 18:57:51 - INFO - app - STEP = 762 | |
| 10/15/2023 18:57:52 - INFO - app - STEP = 763 | |
| 10/15/2023 18:57:53 - INFO - app - STEP = 764 | |
| 10/15/2023 18:57:54 - INFO - app - STEP = 765 | |
| 10/15/2023 18:57:56 - INFO - app - STEP = 766 | |
| 10/15/2023 18:57:57 - INFO - app - STEP = 767 | |
| 10/15/2023 18:57:58 - INFO - app - STEP = 768 | |
| 10/15/2023 18:57:59 - INFO - app - STEP = 769 | |
| 10/15/2023 18:58:01 - INFO - app - STEP = 770 | |
| 10/15/2023 18:58:02 - INFO - app - STEP = 771 | |
| 10/15/2023 18:58:03 - INFO - app - STEP = 772 | |
| 10/15/2023 18:58:05 - INFO - app - STEP = 773 | |
| 10/15/2023 18:58:06 - INFO - app - STEP = 774 | |
| 10/15/2023 18:58:07 - INFO - app - STEP = 775 | |
| 10/15/2023 18:58:09 - INFO - app - STEP = 776 | |
| 10/15/2023 18:58:10 - INFO - app - STEP = 777 | |
| 10/15/2023 18:58:11 - INFO - app - STEP = 778 | |
| 10/15/2023 18:58:12 - INFO - app - STEP = 779 | |
| 10/15/2023 18:58:14 - INFO - app - STEP = 780 | |
| 10/15/2023 18:58:15 - INFO - app - STEP = 781 | |
| 10/15/2023 18:58:16 - INFO - app - STEP = 782 | |
| 10/15/2023 18:58:18 - INFO - app - STEP = 783 | |
| 10/15/2023 18:58:19 - INFO - app - STEP = 784 | |
| 10/15/2023 18:58:20 - INFO - app - STEP = 785 | |
| 10/15/2023 18:58:22 - INFO - app - STEP = 786 | |
| 10/15/2023 18:58:23 - INFO - app - STEP = 787 | |
| 10/15/2023 18:58:24 - INFO - app - STEP = 788 | |
| 10/15/2023 18:58:25 - INFO - app - STEP = 789 | |
| 10/15/2023 18:58:27 - INFO - app - STEP = 790 | |
| 10/15/2023 18:58:28 - INFO - app - STEP = 791 | |
| 10/15/2023 18:58:29 - INFO - app - STEP = 792 | |
| 10/15/2023 18:58:30 - INFO - app - STEP = 793 | |
| 10/15/2023 18:58:32 - INFO - app - STEP = 794 | |
| 10/15/2023 18:58:33 - INFO - app - STEP = 795 | |
| 10/15/2023 18:58:34 - INFO - app - STEP = 796 | |
| 10/15/2023 18:58:36 - INFO - app - STEP = 797 | |
| 10/15/2023 18:58:37 - INFO - app - STEP = 798 | |
| 10/15/2023 18:58:38 - INFO - app - STEP = 799 | |
| 10/15/2023 18:58:40 - INFO - app - STEP = 800 | |
| 10/15/2023 18:58:40 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-800 | |
| 10/15/2023 18:58:40 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-800/optimizer.bin | |
| 10/15/2023 18:58:40 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-800/scheduler.bin | |
| 10/15/2023 18:58:40 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-800/random_states_0.pkl | |
| 10/15/2023 18:58:40 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-800 | |
| 10/15/2023 18:58:41 - INFO - app - STEP = 801 | |
| 10/15/2023 18:58:42 - INFO - app - STEP = 802 | |
| 10/15/2023 18:58:43 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 18:59:09 - INFO - app - STEP = 803 | |
| 10/15/2023 18:59:10 - INFO - app - STEP = 804 | |
| 10/15/2023 18:59:11 - INFO - app - STEP = 805 | |
| 10/15/2023 18:59:13 - INFO - app - STEP = 806 | |
| 10/15/2023 18:59:14 - INFO - app - STEP = 807 | |
| 10/15/2023 18:59:15 - INFO - app - STEP = 808 | |
| 10/15/2023 18:59:17 - INFO - app - STEP = 809 | |
| 10/15/2023 18:59:18 - INFO - app - STEP = 810 | |
| 10/15/2023 18:59:19 - INFO - app - STEP = 811 | |
| 10/15/2023 18:59:21 - INFO - app - STEP = 812 | |
| 10/15/2023 18:59:22 - INFO - app - STEP = 813 | |
| 10/15/2023 18:59:23 - INFO - app - STEP = 814 | |
| 10/15/2023 18:59:25 - INFO - app - STEP = 815 | |
| 10/15/2023 18:59:26 - INFO - app - STEP = 816 | |
| 10/15/2023 18:59:27 - INFO - app - STEP = 817 | |
| 10/15/2023 18:59:29 - INFO - app - STEP = 818 | |
| 10/15/2023 18:59:30 - INFO - app - STEP = 819 | |
| 10/15/2023 18:59:31 - INFO - app - STEP = 820 | |
| 10/15/2023 18:59:32 - INFO - app - STEP = 821 | |
| 10/15/2023 18:59:34 - INFO - app - STEP = 822 | |
| 10/15/2023 18:59:35 - INFO - app - STEP = 823 | |
| 10/15/2023 18:59:36 - INFO - app - STEP = 824 | |
| 10/15/2023 18:59:38 - INFO - app - STEP = 825 | |
| 10/15/2023 18:59:39 - INFO - app - STEP = 826 | |
| 10/15/2023 18:59:40 - INFO - app - STEP = 827 | |
| 10/15/2023 18:59:42 - INFO - app - STEP = 828 | |
| 10/15/2023 18:59:43 - INFO - app - STEP = 829 | |
| 10/15/2023 18:59:44 - INFO - app - STEP = 830 | |
| 10/15/2023 18:59:45 - INFO - app - STEP = 831 | |
| 10/15/2023 18:59:47 - INFO - app - STEP = 832 | |
| 10/15/2023 18:59:48 - INFO - app - STEP = 833 | |
| 10/15/2023 18:59:49 - INFO - app - STEP = 834 | |
| 10/15/2023 18:59:51 - INFO - app - STEP = 835 | |
| 10/15/2023 18:59:52 - INFO - app - STEP = 836 | |
| 10/15/2023 18:59:53 - INFO - app - STEP = 837 | |
| 10/15/2023 18:59:55 - INFO - app - STEP = 838 | |
| 10/15/2023 18:59:56 - INFO - app - STEP = 839 | |
| 10/15/2023 18:59:57 - INFO - app - STEP = 840 | |
| 10/15/2023 18:59:58 - INFO - app - STEP = 841 | |
| 10/15/2023 19:00:00 - INFO - app - STEP = 842 | |
| 10/15/2023 19:00:01 - INFO - app - STEP = 843 | |
| 10/15/2023 19:00:02 - INFO - app - STEP = 844 | |
| 10/15/2023 19:00:04 - INFO - app - STEP = 845 | |
| 10/15/2023 19:00:05 - INFO - app - STEP = 846 | |
| 10/15/2023 19:00:06 - INFO - app - STEP = 847 | |
| 10/15/2023 19:00:08 - INFO - app - STEP = 848 | |
| 10/15/2023 19:00:09 - INFO - app - STEP = 849 | |
| 10/15/2023 19:00:10 - INFO - app - STEP = 850 | |
| 10/15/2023 19:00:11 - INFO - app - STEP = 851 | |
| 10/15/2023 19:00:13 - INFO - app - STEP = 852 | |
| 10/15/2023 19:00:14 - INFO - app - STEP = 853 | |
| 10/15/2023 19:00:15 - INFO - app - STEP = 854 | |
| 10/15/2023 19:00:16 - INFO - app - STEP = 855 | |
| 10/15/2023 19:00:18 - INFO - app - STEP = 856 | |
| 10/15/2023 19:00:19 - INFO - app - STEP = 857 | |
| 10/15/2023 19:00:20 - INFO - app - STEP = 858 | |
| 10/15/2023 19:00:22 - INFO - app - STEP = 859 | |
| 10/15/2023 19:00:23 - INFO - app - STEP = 860 | |
| 10/15/2023 19:00:24 - INFO - app - STEP = 861 | |
| 10/15/2023 19:00:26 - INFO - app - STEP = 862 | |
| 10/15/2023 19:00:27 - INFO - app - STEP = 863 | |
| 10/15/2023 19:00:28 - INFO - app - STEP = 864 | |
| 10/15/2023 19:00:29 - INFO - app - STEP = 865 | |
| 10/15/2023 19:00:31 - INFO - app - STEP = 866 | |
| 10/15/2023 19:00:32 - INFO - app - STEP = 867 | |
| 10/15/2023 19:00:33 - INFO - app - STEP = 868 | |
| 10/15/2023 19:00:35 - INFO - app - STEP = 869 | |
| 10/15/2023 19:00:36 - INFO - app - STEP = 870 | |
| 10/15/2023 19:00:37 - INFO - app - STEP = 871 | |
| 10/15/2023 19:00:39 - INFO - app - STEP = 872 | |
| 10/15/2023 19:00:40 - INFO - app - STEP = 873 | |
| 10/15/2023 19:00:41 - INFO - app - STEP = 874 | |
| 10/15/2023 19:00:42 - INFO - app - STEP = 875 | |
| 10/15/2023 19:00:44 - INFO - app - STEP = 876 | |
| 10/15/2023 19:00:45 - INFO - app - STEP = 877 | |
| 10/15/2023 19:00:46 - INFO - app - STEP = 878 | |
| 10/15/2023 19:00:47 - INFO - app - STEP = 879 | |
| 10/15/2023 19:00:49 - INFO - app - STEP = 880 | |
| 10/15/2023 19:00:50 - INFO - app - STEP = 881 | |
| 10/15/2023 19:00:51 - INFO - app - STEP = 882 | |
| 10/15/2023 19:00:53 - INFO - app - STEP = 883 | |
| 10/15/2023 19:00:54 - INFO - app - STEP = 884 | |
| 10/15/2023 19:00:55 - INFO - app - STEP = 885 | |
| 10/15/2023 19:00:57 - INFO - app - STEP = 886 | |
| 10/15/2023 19:00:58 - INFO - app - STEP = 887 | |
| 10/15/2023 19:00:59 - INFO - app - STEP = 888 | |
| 10/15/2023 19:01:00 - INFO - app - STEP = 889 | |
| 10/15/2023 19:01:02 - INFO - app - STEP = 890 | |
| 10/15/2023 19:01:03 - INFO - app - STEP = 891 | |
| 10/15/2023 19:01:04 - INFO - app - STEP = 892 | |
| 10/15/2023 19:01:05 - INFO - app - STEP = 893 | |
| 10/15/2023 19:01:07 - INFO - app - STEP = 894 | |
| 10/15/2023 19:01:08 - INFO - app - STEP = 895 | |
| 10/15/2023 19:01:09 - INFO - app - STEP = 896 | |
| 10/15/2023 19:01:11 - INFO - app - STEP = 897 | |
| 10/15/2023 19:01:12 - INFO - app - STEP = 898 | |
| 10/15/2023 19:01:13 - INFO - app - STEP = 899 | |
| 10/15/2023 19:01:15 - INFO - app - STEP = 900 | |
| 10/15/2023 19:01:16 - INFO - app - STEP = 901 | |
| 10/15/2023 19:01:17 - INFO - app - STEP = 902 | |
| 10/15/2023 19:01:18 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:01:44 - INFO - app - STEP = 903 | |
| 10/15/2023 19:01:45 - INFO - app - STEP = 904 | |
| 10/15/2023 19:01:46 - INFO - app - STEP = 905 | |
| 10/15/2023 19:01:48 - INFO - app - STEP = 906 | |
| 10/15/2023 19:01:49 - INFO - app - STEP = 907 | |
| 10/15/2023 19:01:50 - INFO - app - STEP = 908 | |
| 10/15/2023 19:01:51 - INFO - app - STEP = 909 | |
| 10/15/2023 19:01:53 - INFO - app - STEP = 910 | |
| 10/15/2023 19:01:54 - INFO - app - STEP = 911 | |
| 10/15/2023 19:01:55 - INFO - app - STEP = 912 | |
| 10/15/2023 19:01:57 - INFO - app - STEP = 913 | |
| 10/15/2023 19:01:58 - INFO - app - STEP = 914 | |
| 10/15/2023 19:01:59 - INFO - app - STEP = 915 | |
| 10/15/2023 19:02:01 - INFO - app - STEP = 916 | |
| 10/15/2023 19:02:02 - INFO - app - STEP = 917 | |
| 10/15/2023 19:02:03 - INFO - app - STEP = 918 | |
| 10/15/2023 19:02:04 - INFO - app - STEP = 919 | |
| 10/15/2023 19:02:06 - INFO - app - STEP = 920 | |
| 10/15/2023 19:02:07 - INFO - app - STEP = 921 | |
| 10/15/2023 19:02:08 - INFO - app - STEP = 922 | |
| 10/15/2023 19:02:10 - INFO - app - STEP = 923 | |
| 10/15/2023 19:02:11 - INFO - app - STEP = 924 | |
| 10/15/2023 19:02:12 - INFO - app - STEP = 925 | |
| 10/15/2023 19:02:14 - INFO - app - STEP = 926 | |
| 10/15/2023 19:02:15 - INFO - app - STEP = 927 | |
| 10/15/2023 19:02:16 - INFO - app - STEP = 928 | |
| 10/15/2023 19:02:18 - INFO - app - STEP = 929 | |
| 10/15/2023 19:02:19 - INFO - app - STEP = 930 | |
| 10/15/2023 19:02:20 - INFO - app - STEP = 931 | |
| 10/15/2023 19:02:22 - INFO - app - STEP = 932 | |
| 10/15/2023 19:02:23 - INFO - app - STEP = 933 | |
| 10/15/2023 19:02:24 - INFO - app - STEP = 934 | |
| 10/15/2023 19:02:25 - INFO - app - STEP = 935 | |
| 10/15/2023 19:02:27 - INFO - app - STEP = 936 | |
| 10/15/2023 19:02:28 - INFO - app - STEP = 937 | |
| 10/15/2023 19:02:29 - INFO - app - STEP = 938 | |
| 10/15/2023 19:02:30 - INFO - app - STEP = 939 | |
| 10/15/2023 19:02:32 - INFO - app - STEP = 940 | |
| 10/15/2023 19:02:33 - INFO - app - STEP = 941 | |
| 10/15/2023 19:02:34 - INFO - app - STEP = 942 | |
| 10/15/2023 19:02:36 - INFO - app - STEP = 943 | |
| 10/15/2023 19:02:37 - INFO - app - STEP = 944 | |
| 10/15/2023 19:02:38 - INFO - app - STEP = 945 | |
| 10/15/2023 19:02:40 - INFO - app - STEP = 946 | |
| 10/15/2023 19:02:41 - INFO - app - STEP = 947 | |
| 10/15/2023 19:02:42 - INFO - app - STEP = 948 | |
| 10/15/2023 19:02:44 - INFO - app - STEP = 949 | |
| 10/15/2023 19:02:45 - INFO - app - STEP = 950 | |
| 10/15/2023 19:02:46 - INFO - app - STEP = 951 | |
| 10/15/2023 19:02:48 - INFO - app - STEP = 952 | |
| 10/15/2023 19:02:49 - INFO - app - STEP = 953 | |
| 10/15/2023 19:02:50 - INFO - app - STEP = 954 | |
| 10/15/2023 19:02:51 - INFO - app - STEP = 955 | |
| 10/15/2023 19:02:53 - INFO - app - STEP = 956 | |
| 10/15/2023 19:02:54 - INFO - app - STEP = 957 | |
| 10/15/2023 19:02:56 - INFO - app - STEP = 958 | |
| 10/15/2023 19:02:57 - INFO - app - STEP = 959 | |
| 10/15/2023 19:02:58 - INFO - app - STEP = 960 | |
| 10/15/2023 19:02:59 - INFO - app - STEP = 961 | |
| 10/15/2023 19:03:01 - INFO - app - STEP = 962 | |
| 10/15/2023 19:03:02 - INFO - app - STEP = 963 | |
| 10/15/2023 19:03:03 - INFO - app - STEP = 964 | |
| 10/15/2023 19:03:05 - INFO - app - STEP = 965 | |
| 10/15/2023 19:03:06 - INFO - app - STEP = 966 | |
| 10/15/2023 19:03:07 - INFO - app - STEP = 967 | |
| 10/15/2023 19:03:09 - INFO - app - STEP = 968 | |
| 10/15/2023 19:03:10 - INFO - app - STEP = 969 | |
| 10/15/2023 19:03:11 - INFO - app - STEP = 970 | |
| 10/15/2023 19:03:12 - INFO - app - STEP = 971 | |
| 10/15/2023 19:03:14 - INFO - app - STEP = 972 | |
| 10/15/2023 19:03:15 - INFO - app - STEP = 973 | |
| 10/15/2023 19:03:16 - INFO - app - STEP = 974 | |
| 10/15/2023 19:03:18 - INFO - app - STEP = 975 | |
| 10/15/2023 19:03:19 - INFO - app - STEP = 976 | |
| 10/15/2023 19:03:20 - INFO - app - STEP = 977 | |
| 10/15/2023 19:03:22 - INFO - app - STEP = 978 | |
| 10/15/2023 19:03:23 - INFO - app - STEP = 979 | |
| 10/15/2023 19:03:24 - INFO - app - STEP = 980 | |
| 10/15/2023 19:03:26 - INFO - app - STEP = 981 | |
| 10/15/2023 19:03:27 - INFO - app - STEP = 982 | |
| 10/15/2023 19:03:28 - INFO - app - STEP = 983 | |
| 10/15/2023 19:03:30 - INFO - app - STEP = 984 | |
| 10/15/2023 19:03:31 - INFO - app - STEP = 985 | |
| 10/15/2023 19:03:32 - INFO - app - STEP = 986 | |
| 10/15/2023 19:03:33 - INFO - app - STEP = 987 | |
| 10/15/2023 19:03:35 - INFO - app - STEP = 988 | |
| 10/15/2023 19:03:36 - INFO - app - STEP = 989 | |
| 10/15/2023 19:03:37 - INFO - app - STEP = 990 | |
| 10/15/2023 19:03:39 - INFO - app - STEP = 991 | |
| 10/15/2023 19:03:40 - INFO - app - STEP = 992 | |
| 10/15/2023 19:03:41 - INFO - app - STEP = 993 | |
| 10/15/2023 19:03:43 - INFO - app - STEP = 994 | |
| 10/15/2023 19:03:44 - INFO - app - STEP = 995 | |
| 10/15/2023 19:03:45 - INFO - app - STEP = 996 | |
| 10/15/2023 19:03:46 - INFO - app - STEP = 997 | |
| 10/15/2023 19:03:48 - INFO - app - STEP = 998 | |
| 10/15/2023 19:03:49 - INFO - app - STEP = 999 | |
| 10/15/2023 19:03:51 - INFO - app - STEP = 1000 | |
| 10/15/2023 19:03:51 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-1000 | |
| 10/15/2023 19:03:51 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-1000/optimizer.bin | |
| 10/15/2023 19:03:51 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-1000/scheduler.bin | |
| 10/15/2023 19:03:51 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-1000/random_states_0.pkl | |
| 10/15/2023 19:03:51 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-1000 | |
| 10/15/2023 19:03:52 - INFO - app - STEP = 1001 | |
| 10/15/2023 19:03:53 - INFO - app - STEP = 1002 | |
| 10/15/2023 19:03:54 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:04:20 - INFO - app - STEP = 1003 | |
| 10/15/2023 19:04:21 - INFO - app - STEP = 1004 | |
| 10/15/2023 19:04:22 - INFO - app - STEP = 1005 | |
| 10/15/2023 19:04:24 - INFO - app - STEP = 1006 | |
| 10/15/2023 19:04:25 - INFO - app - STEP = 1007 | |
| 10/15/2023 19:04:26 - INFO - app - STEP = 1008 | |
| 10/15/2023 19:04:28 - INFO - app - STEP = 1009 | |
| 10/15/2023 19:04:29 - INFO - app - STEP = 1010 | |
| 10/15/2023 19:04:30 - INFO - app - STEP = 1011 | |
| 10/15/2023 19:04:32 - INFO - app - STEP = 1012 | |
| 10/15/2023 19:04:33 - INFO - app - STEP = 1013 | |
| 10/15/2023 19:04:34 - INFO - app - STEP = 1014 | |
| 10/15/2023 19:04:35 - INFO - app - STEP = 1015 | |
| 10/15/2023 19:04:37 - INFO - app - STEP = 1016 | |
| 10/15/2023 19:04:38 - INFO - app - STEP = 1017 | |
| 10/15/2023 19:04:40 - INFO - app - STEP = 1018 | |
| 10/15/2023 19:04:41 - INFO - app - STEP = 1019 | |
| 10/15/2023 19:04:42 - INFO - app - STEP = 1020 | |
| 10/15/2023 19:04:43 - INFO - app - STEP = 1021 | |
| 10/15/2023 19:04:45 - INFO - app - STEP = 1022 | |
| 10/15/2023 19:04:46 - INFO - app - STEP = 1023 | |
| 10/15/2023 19:04:47 - INFO - app - STEP = 1024 | |
| 10/15/2023 19:04:49 - INFO - app - STEP = 1025 | |
| 10/15/2023 19:04:50 - INFO - app - STEP = 1026 | |
| 10/15/2023 19:04:51 - INFO - app - STEP = 1027 | |
| 10/15/2023 19:04:53 - INFO - app - STEP = 1028 | |
| 10/15/2023 19:04:54 - INFO - app - STEP = 1029 | |
| 10/15/2023 19:04:55 - INFO - app - STEP = 1030 | |
| 10/15/2023 19:04:56 - INFO - app - STEP = 1031 | |
| 10/15/2023 19:04:58 - INFO - app - STEP = 1032 | |
| 10/15/2023 19:04:59 - INFO - app - STEP = 1033 | |
| 10/15/2023 19:05:00 - INFO - app - STEP = 1034 | |
| 10/15/2023 19:05:02 - INFO - app - STEP = 1035 | |
| 10/15/2023 19:05:03 - INFO - app - STEP = 1036 | |
| 10/15/2023 19:05:04 - INFO - app - STEP = 1037 | |
| 10/15/2023 19:05:06 - INFO - app - STEP = 1038 | |
| 10/15/2023 19:05:07 - INFO - app - STEP = 1039 | |
| 10/15/2023 19:05:08 - INFO - app - STEP = 1040 | |
| 10/15/2023 19:05:10 - INFO - app - STEP = 1041 | |
| 10/15/2023 19:05:11 - INFO - app - STEP = 1042 | |
| 10/15/2023 19:05:12 - INFO - app - STEP = 1043 | |
| 10/15/2023 19:05:14 - INFO - app - STEP = 1044 | |
| 10/15/2023 19:05:15 - INFO - app - STEP = 1045 | |
| 10/15/2023 19:05:16 - INFO - app - STEP = 1046 | |
| 10/15/2023 19:05:17 - INFO - app - STEP = 1047 | |
| 10/15/2023 19:05:19 - INFO - app - STEP = 1048 | |
| 10/15/2023 19:05:20 - INFO - app - STEP = 1049 | |
| 10/15/2023 19:05:21 - INFO - app - STEP = 1050 | |
| 10/15/2023 19:05:23 - INFO - app - STEP = 1051 | |
| 10/15/2023 19:05:24 - INFO - app - STEP = 1052 | |
| 10/15/2023 19:05:25 - INFO - app - STEP = 1053 | |
| 10/15/2023 19:05:27 - INFO - app - STEP = 1054 | |
| 10/15/2023 19:05:28 - INFO - app - STEP = 1055 | |
| 10/15/2023 19:05:29 - INFO - app - STEP = 1056 | |
| 10/15/2023 19:05:30 - INFO - app - STEP = 1057 | |
| 10/15/2023 19:05:32 - INFO - app - STEP = 1058 | |
| 10/15/2023 19:05:33 - INFO - app - STEP = 1059 | |
| 10/15/2023 19:05:34 - INFO - app - STEP = 1060 | |
| 10/15/2023 19:05:36 - INFO - app - STEP = 1061 | |
| 10/15/2023 19:05:37 - INFO - app - STEP = 1062 | |
| 10/15/2023 19:05:38 - INFO - app - STEP = 1063 | |
| 10/15/2023 19:05:40 - INFO - app - STEP = 1064 | |
| 10/15/2023 19:05:41 - INFO - app - STEP = 1065 | |
| 10/15/2023 19:05:42 - INFO - app - STEP = 1066 | |
| 10/15/2023 19:05:44 - INFO - app - STEP = 1067 | |
| 10/15/2023 19:05:45 - INFO - app - STEP = 1068 | |
| 10/15/2023 19:05:46 - INFO - app - STEP = 1069 | |
| 10/15/2023 19:05:48 - INFO - app - STEP = 1070 | |
| 10/15/2023 19:05:49 - INFO - app - STEP = 1071 | |
| 10/15/2023 19:05:50 - INFO - app - STEP = 1072 | |
| 10/15/2023 19:05:51 - INFO - app - STEP = 1073 | |
| 10/15/2023 19:05:53 - INFO - app - STEP = 1074 | |
| 10/15/2023 19:05:54 - INFO - app - STEP = 1075 | |
| 10/15/2023 19:05:56 - INFO - app - STEP = 1076 | |
| 10/15/2023 19:05:57 - INFO - app - STEP = 1077 | |
| 10/15/2023 19:05:58 - INFO - app - STEP = 1078 | |
| 10/15/2023 19:05:59 - INFO - app - STEP = 1079 | |
| 10/15/2023 19:06:01 - INFO - app - STEP = 1080 | |
| 10/15/2023 19:06:02 - INFO - app - STEP = 1081 | |
| 10/15/2023 19:06:03 - INFO - app - STEP = 1082 | |
| 10/15/2023 19:06:04 - INFO - app - STEP = 1083 | |
| 10/15/2023 19:06:06 - INFO - app - STEP = 1084 | |
| 10/15/2023 19:06:07 - INFO - app - STEP = 1085 | |
| 10/15/2023 19:06:08 - INFO - app - STEP = 1086 | |
| 10/15/2023 19:06:10 - INFO - app - STEP = 1087 | |
| 10/15/2023 19:06:11 - INFO - app - STEP = 1088 | |
| 10/15/2023 19:06:12 - INFO - app - STEP = 1089 | |
| 10/15/2023 19:06:14 - INFO - app - STEP = 1090 | |
| 10/15/2023 19:06:15 - INFO - app - STEP = 1091 | |
| 10/15/2023 19:06:16 - INFO - app - STEP = 1092 | |
| 10/15/2023 19:06:17 - INFO - app - STEP = 1093 | |
| 10/15/2023 19:06:19 - INFO - app - STEP = 1094 | |
| 10/15/2023 19:06:20 - INFO - app - STEP = 1095 | |
| 10/15/2023 19:06:21 - INFO - app - STEP = 1096 | |
| 10/15/2023 19:06:23 - INFO - app - STEP = 1097 | |
| 10/15/2023 19:06:24 - INFO - app - STEP = 1098 | |
| 10/15/2023 19:06:25 - INFO - app - STEP = 1099 | |
| 10/15/2023 19:06:27 - INFO - app - STEP = 1100 | |
| 10/15/2023 19:06:28 - INFO - app - STEP = 1101 | |
| 10/15/2023 19:06:29 - INFO - app - STEP = 1102 | |
| 10/15/2023 19:06:30 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:06:56 - INFO - app - STEP = 1103 | |
| 10/15/2023 19:06:57 - INFO - app - STEP = 1104 | |
| 10/15/2023 19:06:58 - INFO - app - STEP = 1105 | |
| 10/15/2023 19:07:00 - INFO - app - STEP = 1106 | |
| 10/15/2023 19:07:01 - INFO - app - STEP = 1107 | |
| 10/15/2023 19:07:02 - INFO - app - STEP = 1108 | |
| 10/15/2023 19:07:03 - INFO - app - STEP = 1109 | |
| 10/15/2023 19:07:05 - INFO - app - STEP = 1110 | |
| 10/15/2023 19:07:06 - INFO - app - STEP = 1111 | |
| 10/15/2023 19:07:07 - INFO - app - STEP = 1112 | |
| 10/15/2023 19:07:08 - INFO - app - STEP = 1113 | |
| 10/15/2023 19:07:10 - INFO - app - STEP = 1114 | |
| 10/15/2023 19:07:11 - INFO - app - STEP = 1115 | |
| 10/15/2023 19:07:12 - INFO - app - STEP = 1116 | |
| 10/15/2023 19:07:14 - INFO - app - STEP = 1117 | |
| 10/15/2023 19:07:15 - INFO - app - STEP = 1118 | |
| 10/15/2023 19:07:16 - INFO - app - STEP = 1119 | |
| 10/15/2023 19:07:18 - INFO - app - STEP = 1120 | |
| 10/15/2023 19:07:19 - INFO - app - STEP = 1121 | |
| 10/15/2023 19:07:20 - INFO - app - STEP = 1122 | |
| 10/15/2023 19:07:21 - INFO - app - STEP = 1123 | |
| 10/15/2023 19:07:23 - INFO - app - STEP = 1124 | |
| 10/15/2023 19:07:24 - INFO - app - STEP = 1125 | |
| 10/15/2023 19:07:25 - INFO - app - STEP = 1126 | |
| 10/15/2023 19:07:27 - INFO - app - STEP = 1127 | |
| 10/15/2023 19:07:28 - INFO - app - STEP = 1128 | |
| 10/15/2023 19:07:29 - INFO - app - STEP = 1129 | |
| 10/15/2023 19:07:31 - INFO - app - STEP = 1130 | |
| 10/15/2023 19:07:32 - INFO - app - STEP = 1131 | |
| 10/15/2023 19:07:33 - INFO - app - STEP = 1132 | |
| 10/15/2023 19:07:34 - INFO - app - STEP = 1133 | |
| 10/15/2023 19:07:36 - INFO - app - STEP = 1134 | |
| 10/15/2023 19:07:37 - INFO - app - STEP = 1135 | |
| 10/15/2023 19:07:38 - INFO - app - STEP = 1136 | |
| 10/15/2023 19:07:40 - INFO - app - STEP = 1137 | |
| 10/15/2023 19:07:41 - INFO - app - STEP = 1138 | |
| 10/15/2023 19:07:42 - INFO - app - STEP = 1139 | |
| 10/15/2023 19:07:43 - INFO - app - STEP = 1140 | |
| 10/15/2023 19:07:45 - INFO - app - STEP = 1141 | |
| 10/15/2023 19:07:46 - INFO - app - STEP = 1142 | |
| 10/15/2023 19:07:47 - INFO - app - STEP = 1143 | |
| 10/15/2023 19:07:49 - INFO - app - STEP = 1144 | |
| 10/15/2023 19:07:50 - INFO - app - STEP = 1145 | |
| 10/15/2023 19:07:51 - INFO - app - STEP = 1146 | |
| 10/15/2023 19:07:52 - INFO - app - STEP = 1147 | |
| 10/15/2023 19:07:54 - INFO - app - STEP = 1148 | |
| 10/15/2023 19:07:55 - INFO - app - STEP = 1149 | |
| 10/15/2023 19:07:56 - INFO - app - STEP = 1150 | |
| 10/15/2023 19:07:58 - INFO - app - STEP = 1151 | |
| 10/15/2023 19:07:59 - INFO - app - STEP = 1152 | |
| 10/15/2023 19:08:00 - INFO - app - STEP = 1153 | |
| 10/15/2023 19:08:02 - INFO - app - STEP = 1154 | |
| 10/15/2023 19:08:03 - INFO - app - STEP = 1155 | |
| 10/15/2023 19:08:04 - INFO - app - STEP = 1156 | |
| 10/15/2023 19:08:05 - INFO - app - STEP = 1157 | |
| 10/15/2023 19:08:07 - INFO - app - STEP = 1158 | |
| 10/15/2023 19:08:08 - INFO - app - STEP = 1159 | |
| 10/15/2023 19:08:09 - INFO - app - STEP = 1160 | |
| 10/15/2023 19:08:11 - INFO - app - STEP = 1161 | |
| 10/15/2023 19:08:12 - INFO - app - STEP = 1162 | |
| 10/15/2023 19:08:13 - INFO - app - STEP = 1163 | |
| 10/15/2023 19:08:15 - INFO - app - STEP = 1164 | |
| 10/15/2023 19:08:16 - INFO - app - STEP = 1165 | |
| 10/15/2023 19:08:17 - INFO - app - STEP = 1166 | |
| 10/15/2023 19:08:18 - INFO - app - STEP = 1167 | |
| 10/15/2023 19:08:20 - INFO - app - STEP = 1168 | |
| 10/15/2023 19:08:21 - INFO - app - STEP = 1169 | |
| 10/15/2023 19:08:22 - INFO - app - STEP = 1170 | |
| 10/15/2023 19:08:24 - INFO - app - STEP = 1171 | |
| 10/15/2023 19:08:25 - INFO - app - STEP = 1172 | |
| 10/15/2023 19:08:26 - INFO - app - STEP = 1173 | |
| 10/15/2023 19:08:28 - INFO - app - STEP = 1174 | |
| 10/15/2023 19:08:29 - INFO - app - STEP = 1175 | |
| 10/15/2023 19:08:30 - INFO - app - STEP = 1176 | |
| 10/15/2023 19:08:31 - INFO - app - STEP = 1177 | |
| 10/15/2023 19:08:33 - INFO - app - STEP = 1178 | |
| 10/15/2023 19:08:34 - INFO - app - STEP = 1179 | |
| 10/15/2023 19:08:35 - INFO - app - STEP = 1180 | |
| 10/15/2023 19:08:37 - INFO - app - STEP = 1181 | |
| 10/15/2023 19:08:38 - INFO - app - STEP = 1182 | |
| 10/15/2023 19:08:39 - INFO - app - STEP = 1183 | |
| 10/15/2023 19:08:41 - INFO - app - STEP = 1184 | |
| 10/15/2023 19:08:42 - INFO - app - STEP = 1185 | |
| 10/15/2023 19:08:43 - INFO - app - STEP = 1186 | |
| 10/15/2023 19:08:44 - INFO - app - STEP = 1187 | |
| 10/15/2023 19:08:46 - INFO - app - STEP = 1188 | |
| 10/15/2023 19:08:47 - INFO - app - STEP = 1189 | |
| 10/15/2023 19:08:48 - INFO - app - STEP = 1190 | |
| 10/15/2023 19:08:49 - INFO - app - STEP = 1191 | |
| 10/15/2023 19:08:51 - INFO - app - STEP = 1192 | |
| 10/15/2023 19:08:52 - INFO - app - STEP = 1193 | |
| 10/15/2023 19:08:53 - INFO - app - STEP = 1194 | |
| 10/15/2023 19:08:55 - INFO - app - STEP = 1195 | |
| 10/15/2023 19:08:56 - INFO - app - STEP = 1196 | |
| 10/15/2023 19:08:57 - INFO - app - STEP = 1197 | |
| 10/15/2023 19:08:59 - INFO - app - STEP = 1198 | |
| 10/15/2023 19:09:00 - INFO - app - STEP = 1199 | |
| 10/15/2023 19:09:01 - INFO - app - STEP = 1200 | |
| 10/15/2023 19:09:02 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-1200 | |
| 10/15/2023 19:09:02 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-1200/optimizer.bin | |
| 10/15/2023 19:09:02 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-1200/scheduler.bin | |
| 10/15/2023 19:09:02 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-1200/random_states_0.pkl | |
| 10/15/2023 19:09:02 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-1200 | |
| 10/15/2023 19:09:02 - INFO - app - STEP = 1201 | |
| 10/15/2023 19:09:04 - INFO - app - STEP = 1202 | |
| 10/15/2023 19:09:05 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:09:30 - INFO - app - STEP = 1203 | |
| 10/15/2023 19:09:32 - INFO - app - STEP = 1204 | |
| 10/15/2023 19:09:33 - INFO - app - STEP = 1205 | |
| 10/15/2023 19:09:34 - INFO - app - STEP = 1206 | |
| 10/15/2023 19:09:35 - INFO - app - STEP = 1207 | |
| 10/15/2023 19:09:37 - INFO - app - STEP = 1208 | |
| 10/15/2023 19:09:38 - INFO - app - STEP = 1209 | |
| 10/15/2023 19:09:39 - INFO - app - STEP = 1210 | |
| 10/15/2023 19:09:41 - INFO - app - STEP = 1211 | |
| 10/15/2023 19:09:42 - INFO - app - STEP = 1212 | |
| 10/15/2023 19:09:43 - INFO - app - STEP = 1213 | |
| 10/15/2023 19:09:45 - INFO - app - STEP = 1214 | |
| 10/15/2023 19:09:46 - INFO - app - STEP = 1215 | |
| 10/15/2023 19:09:47 - INFO - app - STEP = 1216 | |
| 10/15/2023 19:09:48 - INFO - app - STEP = 1217 | |
| 10/15/2023 19:09:50 - INFO - app - STEP = 1218 | |
| 10/15/2023 19:09:51 - INFO - app - STEP = 1219 | |
| 10/15/2023 19:09:52 - INFO - app - STEP = 1220 | |
| 10/15/2023 19:09:54 - INFO - app - STEP = 1221 | |
| 10/15/2023 19:09:55 - INFO - app - STEP = 1222 | |
| 10/15/2023 19:09:56 - INFO - app - STEP = 1223 | |
| 10/15/2023 19:09:58 - INFO - app - STEP = 1224 | |
| 10/15/2023 19:09:59 - INFO - app - STEP = 1225 | |
| 10/15/2023 19:10:00 - INFO - app - STEP = 1226 | |
| 10/15/2023 19:10:01 - INFO - app - STEP = 1227 | |
| 10/15/2023 19:10:03 - INFO - app - STEP = 1228 | |
| 10/15/2023 19:10:04 - INFO - app - STEP = 1229 | |
| 10/15/2023 19:10:05 - INFO - app - STEP = 1230 | |
| 10/15/2023 19:10:06 - INFO - app - STEP = 1231 | |
| 10/15/2023 19:10:08 - INFO - app - STEP = 1232 | |
| 10/15/2023 19:10:09 - INFO - app - STEP = 1233 | |
| 10/15/2023 19:10:10 - INFO - app - STEP = 1234 | |
| 10/15/2023 19:10:12 - INFO - app - STEP = 1235 | |
| 10/15/2023 19:10:13 - INFO - app - STEP = 1236 | |
| 10/15/2023 19:10:14 - INFO - app - STEP = 1237 | |
| 10/15/2023 19:10:16 - INFO - app - STEP = 1238 | |
| 10/15/2023 19:10:17 - INFO - app - STEP = 1239 | |
| 10/15/2023 19:10:18 - INFO - app - STEP = 1240 | |
| 10/15/2023 19:10:19 - INFO - app - STEP = 1241 | |
| 10/15/2023 19:10:21 - INFO - app - STEP = 1242 | |
| 10/15/2023 19:10:22 - INFO - app - STEP = 1243 | |
| 10/15/2023 19:10:23 - INFO - app - STEP = 1244 | |
| 10/15/2023 19:10:24 - INFO - app - STEP = 1245 | |
| 10/15/2023 19:10:26 - INFO - app - STEP = 1246 | |
| 10/15/2023 19:10:27 - INFO - app - STEP = 1247 | |
| 10/15/2023 19:10:28 - INFO - app - STEP = 1248 | |
| 10/15/2023 19:10:30 - INFO - app - STEP = 1249 | |
| 10/15/2023 19:10:31 - INFO - app - STEP = 1250 | |
| 10/15/2023 19:10:32 - INFO - app - STEP = 1251 | |
| 10/15/2023 19:10:34 - INFO - app - STEP = 1252 | |
| 10/15/2023 19:10:35 - INFO - app - STEP = 1253 | |
| 10/15/2023 19:10:36 - INFO - app - STEP = 1254 | |
| 10/15/2023 19:10:37 - INFO - app - STEP = 1255 | |
| 10/15/2023 19:10:39 - INFO - app - STEP = 1256 | |
| 10/15/2023 19:10:40 - INFO - app - STEP = 1257 | |
| 10/15/2023 19:10:41 - INFO - app - STEP = 1258 | |
| 10/15/2023 19:10:42 - INFO - app - STEP = 1259 | |
| 10/15/2023 19:10:44 - INFO - app - STEP = 1260 | |
| 10/15/2023 19:10:45 - INFO - app - STEP = 1261 | |
| 10/15/2023 19:10:46 - INFO - app - STEP = 1262 | |
| 10/15/2023 19:10:48 - INFO - app - STEP = 1263 | |
| 10/15/2023 19:10:49 - INFO - app - STEP = 1264 | |
| 10/15/2023 19:10:50 - INFO - app - STEP = 1265 | |
| 10/15/2023 19:10:52 - INFO - app - STEP = 1266 | |
| 10/15/2023 19:10:53 - INFO - app - STEP = 1267 | |
| 10/15/2023 19:10:54 - INFO - app - STEP = 1268 | |
| 10/15/2023 19:10:55 - INFO - app - STEP = 1269 | |
| 10/15/2023 19:10:57 - INFO - app - STEP = 1270 | |
| 10/15/2023 19:10:58 - INFO - app - STEP = 1271 | |
| 10/15/2023 19:10:59 - INFO - app - STEP = 1272 | |
| 10/15/2023 19:11:00 - INFO - app - STEP = 1273 | |
| 10/15/2023 19:11:02 - INFO - app - STEP = 1274 | |
| 10/15/2023 19:11:03 - INFO - app - STEP = 1275 | |
| 10/15/2023 19:11:04 - INFO - app - STEP = 1276 | |
| 10/15/2023 19:11:06 - INFO - app - STEP = 1277 | |
| 10/15/2023 19:11:07 - INFO - app - STEP = 1278 | |
| 10/15/2023 19:11:08 - INFO - app - STEP = 1279 | |
| 10/15/2023 19:11:10 - INFO - app - STEP = 1280 | |
| 10/15/2023 19:11:11 - INFO - app - STEP = 1281 | |
| 10/15/2023 19:11:12 - INFO - app - STEP = 1282 | |
| 10/15/2023 19:11:13 - INFO - app - STEP = 1283 | |
| 10/15/2023 19:11:15 - INFO - app - STEP = 1284 | |
| 10/15/2023 19:11:16 - INFO - app - STEP = 1285 | |
| 10/15/2023 19:11:17 - INFO - app - STEP = 1286 | |
| 10/15/2023 19:11:18 - INFO - app - STEP = 1287 | |
| 10/15/2023 19:11:20 - INFO - app - STEP = 1288 | |
| 10/15/2023 19:11:21 - INFO - app - STEP = 1289 | |
| 10/15/2023 19:11:22 - INFO - app - STEP = 1290 | |
| 10/15/2023 19:11:24 - INFO - app - STEP = 1291 | |
| 10/15/2023 19:11:25 - INFO - app - STEP = 1292 | |
| 10/15/2023 19:11:26 - INFO - app - STEP = 1293 | |
| 10/15/2023 19:11:28 - INFO - app - STEP = 1294 | |
| 10/15/2023 19:11:29 - INFO - app - STEP = 1295 | |
| 10/15/2023 19:11:30 - INFO - app - STEP = 1296 | |
| 10/15/2023 19:11:31 - INFO - app - STEP = 1297 | |
| 10/15/2023 19:11:33 - INFO - app - STEP = 1298 | |
| 10/15/2023 19:11:34 - INFO - app - STEP = 1299 | |
| 10/15/2023 19:11:35 - INFO - app - STEP = 1300 | |
| 10/15/2023 19:11:36 - INFO - app - STEP = 1301 | |
| 10/15/2023 19:11:38 - INFO - app - STEP = 1302 | |
| 10/15/2023 19:11:39 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:12:05 - INFO - app - STEP = 1303 | |
| 10/15/2023 19:12:06 - INFO - app - STEP = 1304 | |
| 10/15/2023 19:12:07 - INFO - app - STEP = 1305 | |
| 10/15/2023 19:12:09 - INFO - app - STEP = 1306 | |
| 10/15/2023 19:12:10 - INFO - app - STEP = 1307 | |
| 10/15/2023 19:12:11 - INFO - app - STEP = 1308 | |
| 10/15/2023 19:12:12 - INFO - app - STEP = 1309 | |
| 10/15/2023 19:12:14 - INFO - app - STEP = 1310 | |
| 10/15/2023 19:12:15 - INFO - app - STEP = 1311 | |
| 10/15/2023 19:12:16 - INFO - app - STEP = 1312 | |
| 10/15/2023 19:12:18 - INFO - app - STEP = 1313 | |
| 10/15/2023 19:12:19 - INFO - app - STEP = 1314 | |
| 10/15/2023 19:12:20 - INFO - app - STEP = 1315 | |
| 10/15/2023 19:12:22 - INFO - app - STEP = 1316 | |
| 10/15/2023 19:12:23 - INFO - app - STEP = 1317 | |
| 10/15/2023 19:12:24 - INFO - app - STEP = 1318 | |
| 10/15/2023 19:12:25 - INFO - app - STEP = 1319 | |
| 10/15/2023 19:12:27 - INFO - app - STEP = 1320 | |
| 10/15/2023 19:12:28 - INFO - app - STEP = 1321 | |
| 10/15/2023 19:12:29 - INFO - app - STEP = 1322 | |
| 10/15/2023 19:12:31 - INFO - app - STEP = 1323 | |
| 10/15/2023 19:12:32 - INFO - app - STEP = 1324 | |
| 10/15/2023 19:12:33 - INFO - app - STEP = 1325 | |
| 10/15/2023 19:12:35 - INFO - app - STEP = 1326 | |
| 10/15/2023 19:12:36 - INFO - app - STEP = 1327 | |
| 10/15/2023 19:12:37 - INFO - app - STEP = 1328 | |
| 10/15/2023 19:12:38 - INFO - app - STEP = 1329 | |
| 10/15/2023 19:12:40 - INFO - app - STEP = 1330 | |
| 10/15/2023 19:12:41 - INFO - app - STEP = 1331 | |
| 10/15/2023 19:12:42 - INFO - app - STEP = 1332 | |
| 10/15/2023 19:12:43 - INFO - app - STEP = 1333 | |
| 10/15/2023 19:12:45 - INFO - app - STEP = 1334 | |
| 10/15/2023 19:12:46 - INFO - app - STEP = 1335 | |
| 10/15/2023 19:12:47 - INFO - app - STEP = 1336 | |
| 10/15/2023 19:12:49 - INFO - app - STEP = 1337 | |
| 10/15/2023 19:12:50 - INFO - app - STEP = 1338 | |
| 10/15/2023 19:12:51 - INFO - app - STEP = 1339 | |
| 10/15/2023 19:12:53 - INFO - app - STEP = 1340 | |
| 10/15/2023 19:12:54 - INFO - app - STEP = 1341 | |
| 10/15/2023 19:12:55 - INFO - app - STEP = 1342 | |
| 10/15/2023 19:12:56 - INFO - app - STEP = 1343 | |
| 10/15/2023 19:12:58 - INFO - app - STEP = 1344 | |
| 10/15/2023 19:12:59 - INFO - app - STEP = 1345 | |
| 10/15/2023 19:13:00 - INFO - app - STEP = 1346 | |
| 10/15/2023 19:13:02 - INFO - app - STEP = 1347 | |
| 10/15/2023 19:13:03 - INFO - app - STEP = 1348 | |
| 10/15/2023 19:13:04 - INFO - app - STEP = 1349 | |
| 10/15/2023 19:13:06 - INFO - app - STEP = 1350 | |
| 10/15/2023 19:13:07 - INFO - app - STEP = 1351 | |
| 10/15/2023 19:13:08 - INFO - app - STEP = 1352 | |
| 10/15/2023 19:13:09 - INFO - app - STEP = 1353 | |
| 10/15/2023 19:13:11 - INFO - app - STEP = 1354 | |
| 10/15/2023 19:13:12 - INFO - app - STEP = 1355 | |
| 10/15/2023 19:13:13 - INFO - app - STEP = 1356 | |
| 10/15/2023 19:13:15 - INFO - app - STEP = 1357 | |
| 10/15/2023 19:13:16 - INFO - app - STEP = 1358 | |
| 10/15/2023 19:13:17 - INFO - app - STEP = 1359 | |
| 10/15/2023 19:13:19 - INFO - app - STEP = 1360 | |
| 10/15/2023 19:13:20 - INFO - app - STEP = 1361 | |
| 10/15/2023 19:13:21 - INFO - app - STEP = 1362 | |
| 10/15/2023 19:13:22 - INFO - app - STEP = 1363 | |
| 10/15/2023 19:13:24 - INFO - app - STEP = 1364 | |
| 10/15/2023 19:13:25 - INFO - app - STEP = 1365 | |
| 10/15/2023 19:13:26 - INFO - app - STEP = 1366 | |
| 10/15/2023 19:13:27 - INFO - app - STEP = 1367 | |
| 10/15/2023 19:13:29 - INFO - app - STEP = 1368 | |
| 10/15/2023 19:13:30 - INFO - app - STEP = 1369 | |
| 10/15/2023 19:13:31 - INFO - app - STEP = 1370 | |
| 10/15/2023 19:13:33 - INFO - app - STEP = 1371 | |
| 10/15/2023 19:13:34 - INFO - app - STEP = 1372 | |
| 10/15/2023 19:13:35 - INFO - app - STEP = 1373 | |
| 10/15/2023 19:13:37 - INFO - app - STEP = 1374 | |
| 10/15/2023 19:13:38 - INFO - app - STEP = 1375 | |
| 10/15/2023 19:13:39 - INFO - app - STEP = 1376 | |
| 10/15/2023 19:13:40 - INFO - app - STEP = 1377 | |
| 10/15/2023 19:13:42 - INFO - app - STEP = 1378 | |
| 10/15/2023 19:13:43 - INFO - app - STEP = 1379 | |
| 10/15/2023 19:13:45 - INFO - app - STEP = 1380 | |
| 10/15/2023 19:13:46 - INFO - app - STEP = 1381 | |
| 10/15/2023 19:13:47 - INFO - app - STEP = 1382 | |
| 10/15/2023 19:13:48 - INFO - app - STEP = 1383 | |
| 10/15/2023 19:13:50 - INFO - app - STEP = 1384 | |
| 10/15/2023 19:13:51 - INFO - app - STEP = 1385 | |
| 10/15/2023 19:13:52 - INFO - app - STEP = 1386 | |
| 10/15/2023 19:13:53 - INFO - app - STEP = 1387 | |
| 10/15/2023 19:13:55 - INFO - app - STEP = 1388 | |
| 10/15/2023 19:13:56 - INFO - app - STEP = 1389 | |
| 10/15/2023 19:13:57 - INFO - app - STEP = 1390 | |
| 10/15/2023 19:13:59 - INFO - app - STEP = 1391 | |
| 10/15/2023 19:14:00 - INFO - app - STEP = 1392 | |
| 10/15/2023 19:14:01 - INFO - app - STEP = 1393 | |
| 10/15/2023 19:14:03 - INFO - app - STEP = 1394 | |
| 10/15/2023 19:14:04 - INFO - app - STEP = 1395 | |
| 10/15/2023 19:14:05 - INFO - app - STEP = 1396 | |
| 10/15/2023 19:14:07 - INFO - app - STEP = 1397 | |
| 10/15/2023 19:14:08 - INFO - app - STEP = 1398 | |
| 10/15/2023 19:14:09 - INFO - app - STEP = 1399 | |
| 10/15/2023 19:14:11 - INFO - app - STEP = 1400 | |
| 10/15/2023 19:14:11 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-1400 | |
| 10/15/2023 19:14:11 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-1400/optimizer.bin | |
| 10/15/2023 19:14:11 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-1400/scheduler.bin | |
| 10/15/2023 19:14:11 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-1400/random_states_0.pkl | |
| 10/15/2023 19:14:11 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-1400 | |
| 10/15/2023 19:14:12 - INFO - app - STEP = 1401 | |
| 10/15/2023 19:14:13 - INFO - app - STEP = 1402 | |
| 10/15/2023 19:14:14 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:14:40 - INFO - app - STEP = 1403 | |
| 10/15/2023 19:14:41 - INFO - app - STEP = 1404 | |
| 10/15/2023 19:14:42 - INFO - app - STEP = 1405 | |
| 10/15/2023 19:14:44 - INFO - app - STEP = 1406 | |
| 10/15/2023 19:14:45 - INFO - app - STEP = 1407 | |
| 10/15/2023 19:14:46 - INFO - app - STEP = 1408 | |
| 10/15/2023 19:14:48 - INFO - app - STEP = 1409 | |
| 10/15/2023 19:14:49 - INFO - app - STEP = 1410 | |
| 10/15/2023 19:14:50 - INFO - app - STEP = 1411 | |
| 10/15/2023 19:14:52 - INFO - app - STEP = 1412 | |
| 10/15/2023 19:14:53 - INFO - app - STEP = 1413 | |
| 10/15/2023 19:14:54 - INFO - app - STEP = 1414 | |
| 10/15/2023 19:14:55 - INFO - app - STEP = 1415 | |
| 10/15/2023 19:14:57 - INFO - app - STEP = 1416 | |
| 10/15/2023 19:14:58 - INFO - app - STEP = 1417 | |
| 10/15/2023 19:14:59 - INFO - app - STEP = 1418 | |
| 10/15/2023 19:15:00 - INFO - app - STEP = 1419 | |
| 10/15/2023 19:15:02 - INFO - app - STEP = 1420 | |
| 10/15/2023 19:15:03 - INFO - app - STEP = 1421 | |
| 10/15/2023 19:15:04 - INFO - app - STEP = 1422 | |
| 10/15/2023 19:15:06 - INFO - app - STEP = 1423 | |
| 10/15/2023 19:15:07 - INFO - app - STEP = 1424 | |
| 10/15/2023 19:15:08 - INFO - app - STEP = 1425 | |
| 10/15/2023 19:15:10 - INFO - app - STEP = 1426 | |
| 10/15/2023 19:15:11 - INFO - app - STEP = 1427 | |
| 10/15/2023 19:15:12 - INFO - app - STEP = 1428 | |
| 10/15/2023 19:15:13 - INFO - app - STEP = 1429 | |
| 10/15/2023 19:15:15 - INFO - app - STEP = 1430 | |
| 10/15/2023 19:15:16 - INFO - app - STEP = 1431 | |
| 10/15/2023 19:15:17 - INFO - app - STEP = 1432 | |
| 10/15/2023 19:15:19 - INFO - app - STEP = 1433 | |
| 10/15/2023 19:15:20 - INFO - app - STEP = 1434 | |
| 10/15/2023 19:15:21 - INFO - app - STEP = 1435 | |
| 10/15/2023 19:15:23 - INFO - app - STEP = 1436 | |
| 10/15/2023 19:15:24 - INFO - app - STEP = 1437 | |
| 10/15/2023 19:15:25 - INFO - app - STEP = 1438 | |
| 10/15/2023 19:15:26 - INFO - app - STEP = 1439 | |
| 10/15/2023 19:15:28 - INFO - app - STEP = 1440 | |
| 10/15/2023 19:15:29 - INFO - app - STEP = 1441 | |
| 10/15/2023 19:15:30 - INFO - app - STEP = 1442 | |
| 10/15/2023 19:15:31 - INFO - app - STEP = 1443 | |
| 10/15/2023 19:15:33 - INFO - app - STEP = 1444 | |
| 10/15/2023 19:15:34 - INFO - app - STEP = 1445 | |
| 10/15/2023 19:15:35 - INFO - app - STEP = 1446 | |
| 10/15/2023 19:15:37 - INFO - app - STEP = 1447 | |
| 10/15/2023 19:15:38 - INFO - app - STEP = 1448 | |
| 10/15/2023 19:15:39 - INFO - app - STEP = 1449 | |
| 10/15/2023 19:15:41 - INFO - app - STEP = 1450 | |
| 10/15/2023 19:15:42 - INFO - app - STEP = 1451 | |
| 10/15/2023 19:15:43 - INFO - app - STEP = 1452 | |
| 10/15/2023 19:15:45 - INFO - app - STEP = 1453 | |
| 10/15/2023 19:15:46 - INFO - app - STEP = 1454 | |
| 10/15/2023 19:15:47 - INFO - app - STEP = 1455 | |
| 10/15/2023 19:15:49 - INFO - app - STEP = 1456 | |
| 10/15/2023 19:15:50 - INFO - app - STEP = 1457 | |
| 10/15/2023 19:15:51 - INFO - app - STEP = 1458 | |
| 10/15/2023 19:15:52 - INFO - app - STEP = 1459 | |
| 10/15/2023 19:15:54 - INFO - app - STEP = 1460 | |
| 10/15/2023 19:15:55 - INFO - app - STEP = 1461 | |
| 10/15/2023 19:15:56 - INFO - app - STEP = 1462 | |
| 10/15/2023 19:15:58 - INFO - app - STEP = 1463 | |
| 10/15/2023 19:15:59 - INFO - app - STEP = 1464 | |
| 10/15/2023 19:16:00 - INFO - app - STEP = 1465 | |
| 10/15/2023 19:16:02 - INFO - app - STEP = 1466 | |
| 10/15/2023 19:16:03 - INFO - app - STEP = 1467 | |
| 10/15/2023 19:16:04 - INFO - app - STEP = 1468 | |
| 10/15/2023 19:16:05 - INFO - app - STEP = 1469 | |
| 10/15/2023 19:16:07 - INFO - app - STEP = 1470 | |
| 10/15/2023 19:16:08 - INFO - app - STEP = 1471 | |
| 10/15/2023 19:16:09 - INFO - app - STEP = 1472 | |
| 10/15/2023 19:16:11 - INFO - app - STEP = 1473 | |
| 10/15/2023 19:16:12 - INFO - app - STEP = 1474 | |
| 10/15/2023 19:16:13 - INFO - app - STEP = 1475 | |
| 10/15/2023 19:16:15 - INFO - app - STEP = 1476 | |
| 10/15/2023 19:16:16 - INFO - app - STEP = 1477 | |
| 10/15/2023 19:16:17 - INFO - app - STEP = 1478 | |
| 10/15/2023 19:16:18 - INFO - app - STEP = 1479 | |
| 10/15/2023 19:16:20 - INFO - app - STEP = 1480 | |
| 10/15/2023 19:16:21 - INFO - app - STEP = 1481 | |
| 10/15/2023 19:16:22 - INFO - app - STEP = 1482 | |
| 10/15/2023 19:16:24 - INFO - app - STEP = 1483 | |
| 10/15/2023 19:16:25 - INFO - app - STEP = 1484 | |
| 10/15/2023 19:16:26 - INFO - app - STEP = 1485 | |
| 10/15/2023 19:16:28 - INFO - app - STEP = 1486 | |
| 10/15/2023 19:16:29 - INFO - app - STEP = 1487 | |
| 10/15/2023 19:16:30 - INFO - app - STEP = 1488 | |
| 10/15/2023 19:16:31 - INFO - app - STEP = 1489 | |
| 10/15/2023 19:16:33 - INFO - app - STEP = 1490 | |
| 10/15/2023 19:16:34 - INFO - app - STEP = 1491 | |
| 10/15/2023 19:16:35 - INFO - app - STEP = 1492 | |
| 10/15/2023 19:16:37 - INFO - app - STEP = 1493 | |
| 10/15/2023 19:16:38 - INFO - app - STEP = 1494 | |
| 10/15/2023 19:16:39 - INFO - app - STEP = 1495 | |
| 10/15/2023 19:16:41 - INFO - app - STEP = 1496 | |
| 10/15/2023 19:16:42 - INFO - app - STEP = 1497 | |
| 10/15/2023 19:16:43 - INFO - app - STEP = 1498 | |
| 10/15/2023 19:16:44 - INFO - app - STEP = 1499 | |
| 10/15/2023 19:16:46 - INFO - app - STEP = 1500 | |
| 10/15/2023 19:16:47 - INFO - app - STEP = 1501 | |
| 10/15/2023 19:16:48 - INFO - app - STEP = 1502 | |
| 10/15/2023 19:16:49 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:17:15 - INFO - app - STEP = 1503 | |
| 10/15/2023 19:17:17 - INFO - app - STEP = 1504 | |
| 10/15/2023 19:17:18 - INFO - app - STEP = 1505 | |
| 10/15/2023 19:17:19 - INFO - app - STEP = 1506 | |
| 10/15/2023 19:17:21 - INFO - app - STEP = 1507 | |
| 10/15/2023 19:17:22 - INFO - app - STEP = 1508 | |
| 10/15/2023 19:17:23 - INFO - app - STEP = 1509 | |
| 10/15/2023 19:17:25 - INFO - app - STEP = 1510 | |
| 10/15/2023 19:17:26 - INFO - app - STEP = 1511 | |
| 10/15/2023 19:17:27 - INFO - app - STEP = 1512 | |
| 10/15/2023 19:17:28 - INFO - app - STEP = 1513 | |
| 10/15/2023 19:17:30 - INFO - app - STEP = 1514 | |
| 10/15/2023 19:17:31 - INFO - app - STEP = 1515 | |
| 10/15/2023 19:17:32 - INFO - app - STEP = 1516 | |
| 10/15/2023 19:17:34 - INFO - app - STEP = 1517 | |
| 10/15/2023 19:17:35 - INFO - app - STEP = 1518 | |
| 10/15/2023 19:17:36 - INFO - app - STEP = 1519 | |
| 10/15/2023 19:17:38 - INFO - app - STEP = 1520 | |
| 10/15/2023 19:17:39 - INFO - app - STEP = 1521 | |
| 10/15/2023 19:17:40 - INFO - app - STEP = 1522 | |
| 10/15/2023 19:17:41 - INFO - app - STEP = 1523 | |
| 10/15/2023 19:17:43 - INFO - app - STEP = 1524 | |
| 10/15/2023 19:17:44 - INFO - app - STEP = 1525 | |
| 10/15/2023 19:17:45 - INFO - app - STEP = 1526 | |
| 10/15/2023 19:17:47 - INFO - app - STEP = 1527 | |
| 10/15/2023 19:17:48 - INFO - app - STEP = 1528 | |
| 10/15/2023 19:17:49 - INFO - app - STEP = 1529 | |
| 10/15/2023 19:17:51 - INFO - app - STEP = 1530 | |
| 10/15/2023 19:17:52 - INFO - app - STEP = 1531 | |
| 10/15/2023 19:17:53 - INFO - app - STEP = 1532 | |
| 10/15/2023 19:17:54 - INFO - app - STEP = 1533 | |
| 10/15/2023 19:17:56 - INFO - app - STEP = 1534 | |
| 10/15/2023 19:17:57 - INFO - app - STEP = 1535 | |
| 10/15/2023 19:17:58 - INFO - app - STEP = 1536 | |
| 10/15/2023 19:18:00 - INFO - app - STEP = 1537 | |
| 10/15/2023 19:18:01 - INFO - app - STEP = 1538 | |
| 10/15/2023 19:18:02 - INFO - app - STEP = 1539 | |
| 10/15/2023 19:18:04 - INFO - app - STEP = 1540 | |
| 10/15/2023 19:18:05 - INFO - app - STEP = 1541 | |
| 10/15/2023 19:18:06 - INFO - app - STEP = 1542 | |
| 10/15/2023 19:18:07 - INFO - app - STEP = 1543 | |
| 10/15/2023 19:18:09 - INFO - app - STEP = 1544 | |
| 10/15/2023 19:18:10 - INFO - app - STEP = 1545 | |
| 10/15/2023 19:18:11 - INFO - app - STEP = 1546 | |
| 10/15/2023 19:18:13 - INFO - app - STEP = 1547 | |
| 10/15/2023 19:18:14 - INFO - app - STEP = 1548 | |
| 10/15/2023 19:18:15 - INFO - app - STEP = 1549 | |
| 10/15/2023 19:18:16 - INFO - app - STEP = 1550 | |
| 10/15/2023 19:18:18 - INFO - app - STEP = 1551 | |
| 10/15/2023 19:18:19 - INFO - app - STEP = 1552 | |
| 10/15/2023 19:18:20 - INFO - app - STEP = 1553 | |
| 10/15/2023 19:18:22 - INFO - app - STEP = 1554 | |
| 10/15/2023 19:18:23 - INFO - app - STEP = 1555 | |
| 10/15/2023 19:18:24 - INFO - app - STEP = 1556 | |
| 10/15/2023 19:18:25 - INFO - app - STEP = 1557 | |
| 10/15/2023 19:18:27 - INFO - app - STEP = 1558 | |
| 10/15/2023 19:18:28 - INFO - app - STEP = 1559 | |
| 10/15/2023 19:18:29 - INFO - app - STEP = 1560 | |
| 10/15/2023 19:18:31 - INFO - app - STEP = 1561 | |
| 10/15/2023 19:18:32 - INFO - app - STEP = 1562 | |
| 10/15/2023 19:18:33 - INFO - app - STEP = 1563 | |
| 10/15/2023 19:18:35 - INFO - app - STEP = 1564 | |
| 10/15/2023 19:18:36 - INFO - app - STEP = 1565 | |
| 10/15/2023 19:18:37 - INFO - app - STEP = 1566 | |
| 10/15/2023 19:18:38 - INFO - app - STEP = 1567 | |
| 10/15/2023 19:18:40 - INFO - app - STEP = 1568 | |
| 10/15/2023 19:18:41 - INFO - app - STEP = 1569 | |
| 10/15/2023 19:18:42 - INFO - app - STEP = 1570 | |
| 10/15/2023 19:18:43 - INFO - app - STEP = 1571 | |
| 10/15/2023 19:18:45 - INFO - app - STEP = 1572 | |
| 10/15/2023 19:18:46 - INFO - app - STEP = 1573 | |
| 10/15/2023 19:18:47 - INFO - app - STEP = 1574 | |
| 10/15/2023 19:18:49 - INFO - app - STEP = 1575 | |
| 10/15/2023 19:18:50 - INFO - app - STEP = 1576 | |
| 10/15/2023 19:18:51 - INFO - app - STEP = 1577 | |
| 10/15/2023 19:18:53 - INFO - app - STEP = 1578 | |
| 10/15/2023 19:18:54 - INFO - app - STEP = 1579 | |
| 10/15/2023 19:18:55 - INFO - app - STEP = 1580 | |
| 10/15/2023 19:18:56 - INFO - app - STEP = 1581 | |
| 10/15/2023 19:18:58 - INFO - app - STEP = 1582 | |
| 10/15/2023 19:18:59 - INFO - app - STEP = 1583 | |
| 10/15/2023 19:19:00 - INFO - app - STEP = 1584 | |
| 10/15/2023 19:19:02 - INFO - app - STEP = 1585 | |
| 10/15/2023 19:19:03 - INFO - app - STEP = 1586 | |
| 10/15/2023 19:19:04 - INFO - app - STEP = 1587 | |
| 10/15/2023 19:19:06 - INFO - app - STEP = 1588 | |
| 10/15/2023 19:19:07 - INFO - app - STEP = 1589 | |
| 10/15/2023 19:19:08 - INFO - app - STEP = 1590 | |
| 10/15/2023 19:19:09 - INFO - app - STEP = 1591 | |
| 10/15/2023 19:19:11 - INFO - app - STEP = 1592 | |
| 10/15/2023 19:19:12 - INFO - app - STEP = 1593 | |
| 10/15/2023 19:19:13 - INFO - app - STEP = 1594 | |
| 10/15/2023 19:19:15 - INFO - app - STEP = 1595 | |
| 10/15/2023 19:19:16 - INFO - app - STEP = 1596 | |
| 10/15/2023 19:19:17 - INFO - app - STEP = 1597 | |
| 10/15/2023 19:19:19 - INFO - app - STEP = 1598 | |
| 10/15/2023 19:19:20 - INFO - app - STEP = 1599 | |
| 10/15/2023 19:19:21 - INFO - app - STEP = 1600 | |
| 10/15/2023 19:19:22 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-1600 | |
| 10/15/2023 19:19:22 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-1600/optimizer.bin | |
| 10/15/2023 19:19:22 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-1600/scheduler.bin | |
| 10/15/2023 19:19:22 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-1600/random_states_0.pkl | |
| 10/15/2023 19:19:22 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-1600 | |
| 10/15/2023 19:19:22 - INFO - app - STEP = 1601 | |
| 10/15/2023 19:19:24 - INFO - app - STEP = 1602 | |
| 10/15/2023 19:19:24 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:19:51 - INFO - app - STEP = 1603 | |
| 10/15/2023 19:19:52 - INFO - app - STEP = 1604 | |
| 10/15/2023 19:19:53 - INFO - app - STEP = 1605 | |
| 10/15/2023 19:19:55 - INFO - app - STEP = 1606 | |
| 10/15/2023 19:19:56 - INFO - app - STEP = 1607 | |
| 10/15/2023 19:19:57 - INFO - app - STEP = 1608 | |
| 10/15/2023 19:19:58 - INFO - app - STEP = 1609 | |
| 10/15/2023 19:20:00 - INFO - app - STEP = 1610 | |
| 10/15/2023 19:20:01 - INFO - app - STEP = 1611 | |
| 10/15/2023 19:20:02 - INFO - app - STEP = 1612 | |
| 10/15/2023 19:20:04 - INFO - app - STEP = 1613 | |
| 10/15/2023 19:20:05 - INFO - app - STEP = 1614 | |
| 10/15/2023 19:20:06 - INFO - app - STEP = 1615 | |
| 10/15/2023 19:20:08 - INFO - app - STEP = 1616 | |
| 10/15/2023 19:20:09 - INFO - app - STEP = 1617 | |
| 10/15/2023 19:20:10 - INFO - app - STEP = 1618 | |
| 10/15/2023 19:20:12 - INFO - app - STEP = 1619 | |
| 10/15/2023 19:20:13 - INFO - app - STEP = 1620 | |
| 10/15/2023 19:20:14 - INFO - app - STEP = 1621 | |
| 10/15/2023 19:20:16 - INFO - app - STEP = 1622 | |
| 10/15/2023 19:20:17 - INFO - app - STEP = 1623 | |
| 10/15/2023 19:20:18 - INFO - app - STEP = 1624 | |
| 10/15/2023 19:20:19 - INFO - app - STEP = 1625 | |
| 10/15/2023 19:20:21 - INFO - app - STEP = 1626 | |
| 10/15/2023 19:20:22 - INFO - app - STEP = 1627 | |
| 10/15/2023 19:20:23 - INFO - app - STEP = 1628 | |
| 10/15/2023 19:20:25 - INFO - app - STEP = 1629 | |
| 10/15/2023 19:20:26 - INFO - app - STEP = 1630 | |
| 10/15/2023 19:20:27 - INFO - app - STEP = 1631 | |
| 10/15/2023 19:20:29 - INFO - app - STEP = 1632 | |
| 10/15/2023 19:20:30 - INFO - app - STEP = 1633 | |
| 10/15/2023 19:20:31 - INFO - app - STEP = 1634 | |
| 10/15/2023 19:20:32 - INFO - app - STEP = 1635 | |
| 10/15/2023 19:20:34 - INFO - app - STEP = 1636 | |
| 10/15/2023 19:20:35 - INFO - app - STEP = 1637 | |
| 10/15/2023 19:20:36 - INFO - app - STEP = 1638 | |
| 10/15/2023 19:20:38 - INFO - app - STEP = 1639 | |
| 10/15/2023 19:20:39 - INFO - app - STEP = 1640 | |
| 10/15/2023 19:20:40 - INFO - app - STEP = 1641 | |
| 10/15/2023 19:20:42 - INFO - app - STEP = 1642 | |
| 10/15/2023 19:20:43 - INFO - app - STEP = 1643 | |
| 10/15/2023 19:20:44 - INFO - app - STEP = 1644 | |
| 10/15/2023 19:20:45 - INFO - app - STEP = 1645 | |
| 10/15/2023 19:20:47 - INFO - app - STEP = 1646 | |
| 10/15/2023 19:20:48 - INFO - app - STEP = 1647 | |
| 10/15/2023 19:20:49 - INFO - app - STEP = 1648 | |
| 10/15/2023 19:20:51 - INFO - app - STEP = 1649 | |
| 10/15/2023 19:20:52 - INFO - app - STEP = 1650 | |
| 10/15/2023 19:20:53 - INFO - app - STEP = 1651 | |
| 10/15/2023 19:20:55 - INFO - app - STEP = 1652 | |
| 10/15/2023 19:20:56 - INFO - app - STEP = 1653 | |
| 10/15/2023 19:20:57 - INFO - app - STEP = 1654 | |
| 10/15/2023 19:20:58 - INFO - app - STEP = 1655 | |
| 10/15/2023 19:21:00 - INFO - app - STEP = 1656 | |
| 10/15/2023 19:21:01 - INFO - app - STEP = 1657 | |
| 10/15/2023 19:21:03 - INFO - app - STEP = 1658 | |
| 10/15/2023 19:21:04 - INFO - app - STEP = 1659 | |
| 10/15/2023 19:21:05 - INFO - app - STEP = 1660 | |
| 10/15/2023 19:21:06 - INFO - app - STEP = 1661 | |
| 10/15/2023 19:21:08 - INFO - app - STEP = 1662 | |
| 10/15/2023 19:21:09 - INFO - app - STEP = 1663 | |
| 10/15/2023 19:21:10 - INFO - app - STEP = 1664 | |
| 10/15/2023 19:21:12 - INFO - app - STEP = 1665 | |
| 10/15/2023 19:21:13 - INFO - app - STEP = 1666 | |
| 10/15/2023 19:21:14 - INFO - app - STEP = 1667 | |
| 10/15/2023 19:21:16 - INFO - app - STEP = 1668 | |
| 10/15/2023 19:21:17 - INFO - app - STEP = 1669 | |
| 10/15/2023 19:21:18 - INFO - app - STEP = 1670 | |
| 10/15/2023 19:21:19 - INFO - app - STEP = 1671 | |
| 10/15/2023 19:21:21 - INFO - app - STEP = 1672 | |
| 10/15/2023 19:21:22 - INFO - app - STEP = 1673 | |
| 10/15/2023 19:21:23 - INFO - app - STEP = 1674 | |
| 10/15/2023 19:21:25 - INFO - app - STEP = 1675 | |
| 10/15/2023 19:21:26 - INFO - app - STEP = 1676 | |
| 10/15/2023 19:21:27 - INFO - app - STEP = 1677 | |
| 10/15/2023 19:21:29 - INFO - app - STEP = 1678 | |
| 10/15/2023 19:21:30 - INFO - app - STEP = 1679 | |
| 10/15/2023 19:21:31 - INFO - app - STEP = 1680 | |
| 10/15/2023 19:21:33 - INFO - app - STEP = 1681 | |
| 10/15/2023 19:21:34 - INFO - app - STEP = 1682 | |
| 10/15/2023 19:21:35 - INFO - app - STEP = 1683 | |
| 10/15/2023 19:21:37 - INFO - app - STEP = 1684 | |
| 10/15/2023 19:21:38 - INFO - app - STEP = 1685 | |
| 10/15/2023 19:21:39 - INFO - app - STEP = 1686 | |
| 10/15/2023 19:21:40 - INFO - app - STEP = 1687 | |
| 10/15/2023 19:21:42 - INFO - app - STEP = 1688 | |
| 10/15/2023 19:21:43 - INFO - app - STEP = 1689 | |
| 10/15/2023 19:21:44 - INFO - app - STEP = 1690 | |
| 10/15/2023 19:21:46 - INFO - app - STEP = 1691 | |
| 10/15/2023 19:21:47 - INFO - app - STEP = 1692 | |
| 10/15/2023 19:21:48 - INFO - app - STEP = 1693 | |
| 10/15/2023 19:21:50 - INFO - app - STEP = 1694 | |
| 10/15/2023 19:21:51 - INFO - app - STEP = 1695 | |
| 10/15/2023 19:21:52 - INFO - app - STEP = 1696 | |
| 10/15/2023 19:21:53 - INFO - app - STEP = 1697 | |
| 10/15/2023 19:21:55 - INFO - app - STEP = 1698 | |
| 10/15/2023 19:21:56 - INFO - app - STEP = 1699 | |
| 10/15/2023 19:21:57 - INFO - app - STEP = 1700 | |
| 10/15/2023 19:21:59 - INFO - app - STEP = 1701 | |
| 10/15/2023 19:22:00 - INFO - app - STEP = 1702 | |
| 10/15/2023 19:22:01 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:22:27 - INFO - app - STEP = 1703 | |
| 10/15/2023 19:22:28 - INFO - app - STEP = 1704 | |
| 10/15/2023 19:22:29 - INFO - app - STEP = 1705 | |
| 10/15/2023 19:22:31 - INFO - app - STEP = 1706 | |
| 10/15/2023 19:22:32 - INFO - app - STEP = 1707 | |
| 10/15/2023 19:22:34 - INFO - app - STEP = 1708 | |
| 10/15/2023 19:22:35 - INFO - app - STEP = 1709 | |
| 10/15/2023 19:22:36 - INFO - app - STEP = 1710 | |
| 10/15/2023 19:22:37 - INFO - app - STEP = 1711 | |
| 10/15/2023 19:22:39 - INFO - app - STEP = 1712 | |
| 10/15/2023 19:22:40 - INFO - app - STEP = 1713 | |
| 10/15/2023 19:22:41 - INFO - app - STEP = 1714 | |
| 10/15/2023 19:22:42 - INFO - app - STEP = 1715 | |
| 10/15/2023 19:22:44 - INFO - app - STEP = 1716 | |
| 10/15/2023 19:22:45 - INFO - app - STEP = 1717 | |
| 10/15/2023 19:22:47 - INFO - app - STEP = 1718 | |
| 10/15/2023 19:22:48 - INFO - app - STEP = 1719 | |
| 10/15/2023 19:22:49 - INFO - app - STEP = 1720 | |
| 10/15/2023 19:22:50 - INFO - app - STEP = 1721 | |
| 10/15/2023 19:22:52 - INFO - app - STEP = 1722 | |
| 10/15/2023 19:22:53 - INFO - app - STEP = 1723 | |
| 10/15/2023 19:22:54 - INFO - app - STEP = 1724 | |
| 10/15/2023 19:22:56 - INFO - app - STEP = 1725 | |
| 10/15/2023 19:22:57 - INFO - app - STEP = 1726 | |
| 10/15/2023 19:22:58 - INFO - app - STEP = 1727 | |
| 10/15/2023 19:23:00 - INFO - app - STEP = 1728 | |
| 10/15/2023 19:23:01 - INFO - app - STEP = 1729 | |
| 10/15/2023 19:23:02 - INFO - app - STEP = 1730 | |
| 10/15/2023 19:23:03 - INFO - app - STEP = 1731 | |
| 10/15/2023 19:23:05 - INFO - app - STEP = 1732 | |
| 10/15/2023 19:23:06 - INFO - app - STEP = 1733 | |
| 10/15/2023 19:23:08 - INFO - app - STEP = 1734 | |
| 10/15/2023 19:23:09 - INFO - app - STEP = 1735 | |
| 10/15/2023 19:23:10 - INFO - app - STEP = 1736 | |
| 10/15/2023 19:23:11 - INFO - app - STEP = 1737 | |
| 10/15/2023 19:23:13 - INFO - app - STEP = 1738 | |
| 10/15/2023 19:23:14 - INFO - app - STEP = 1739 | |
| 10/15/2023 19:23:15 - INFO - app - STEP = 1740 | |
| 10/15/2023 19:23:17 - INFO - app - STEP = 1741 | |
| 10/15/2023 19:23:18 - INFO - app - STEP = 1742 | |
| 10/15/2023 19:23:19 - INFO - app - STEP = 1743 | |
| 10/15/2023 19:23:21 - INFO - app - STEP = 1744 | |
| 10/15/2023 19:23:22 - INFO - app - STEP = 1745 | |
| 10/15/2023 19:23:23 - INFO - app - STEP = 1746 | |
| 10/15/2023 19:23:24 - INFO - app - STEP = 1747 | |
| 10/15/2023 19:23:26 - INFO - app - STEP = 1748 | |
| 10/15/2023 19:23:27 - INFO - app - STEP = 1749 | |
| 10/15/2023 19:23:28 - INFO - app - STEP = 1750 | |
| 10/15/2023 19:23:30 - INFO - app - STEP = 1751 | |
| 10/15/2023 19:23:31 - INFO - app - STEP = 1752 | |
| 10/15/2023 19:23:32 - INFO - app - STEP = 1753 | |
| 10/15/2023 19:23:34 - INFO - app - STEP = 1754 | |
| 10/15/2023 19:23:35 - INFO - app - STEP = 1755 | |
| 10/15/2023 19:23:36 - INFO - app - STEP = 1756 | |
| 10/15/2023 19:23:37 - INFO - app - STEP = 1757 | |
| 10/15/2023 19:23:39 - INFO - app - STEP = 1758 | |
| 10/15/2023 19:23:40 - INFO - app - STEP = 1759 | |
| 10/15/2023 19:23:42 - INFO - app - STEP = 1760 | |
| 10/15/2023 19:23:43 - INFO - app - STEP = 1761 | |
| 10/15/2023 19:23:44 - INFO - app - STEP = 1762 | |
| 10/15/2023 19:23:45 - INFO - app - STEP = 1763 | |
| 10/15/2023 19:23:47 - INFO - app - STEP = 1764 | |
| 10/15/2023 19:23:48 - INFO - app - STEP = 1765 | |
| 10/15/2023 19:23:49 - INFO - app - STEP = 1766 | |
| 10/15/2023 19:23:51 - INFO - app - STEP = 1767 | |
| 10/15/2023 19:23:52 - INFO - app - STEP = 1768 | |
| 10/15/2023 19:23:53 - INFO - app - STEP = 1769 | |
| 10/15/2023 19:23:55 - INFO - app - STEP = 1770 | |
| 10/15/2023 19:23:56 - INFO - app - STEP = 1771 | |
| 10/15/2023 19:23:57 - INFO - app - STEP = 1772 | |
| 10/15/2023 19:23:58 - INFO - app - STEP = 1773 | |
| 10/15/2023 19:24:00 - INFO - app - STEP = 1774 | |
| 10/15/2023 19:24:01 - INFO - app - STEP = 1775 | |
| 10/15/2023 19:24:02 - INFO - app - STEP = 1776 | |
| 10/15/2023 19:24:04 - INFO - app - STEP = 1777 | |
| 10/15/2023 19:24:05 - INFO - app - STEP = 1778 | |
| 10/15/2023 19:24:06 - INFO - app - STEP = 1779 | |
| 10/15/2023 19:24:08 - INFO - app - STEP = 1780 | |
| 10/15/2023 19:24:09 - INFO - app - STEP = 1781 | |
| 10/15/2023 19:24:10 - INFO - app - STEP = 1782 | |
| 10/15/2023 19:24:11 - INFO - app - STEP = 1783 | |
| 10/15/2023 19:24:13 - INFO - app - STEP = 1784 | |
| 10/15/2023 19:24:14 - INFO - app - STEP = 1785 | |
| 10/15/2023 19:24:15 - INFO - app - STEP = 1786 | |
| 10/15/2023 19:24:17 - INFO - app - STEP = 1787 | |
| 10/15/2023 19:24:18 - INFO - app - STEP = 1788 | |
| 10/15/2023 19:24:19 - INFO - app - STEP = 1789 | |
| 10/15/2023 19:24:21 - INFO - app - STEP = 1790 | |
| 10/15/2023 19:24:22 - INFO - app - STEP = 1791 | |
| 10/15/2023 19:24:23 - INFO - app - STEP = 1792 | |
| 10/15/2023 19:24:24 - INFO - app - STEP = 1793 | |
| 10/15/2023 19:24:26 - INFO - app - STEP = 1794 | |
| 10/15/2023 19:24:27 - INFO - app - STEP = 1795 | |
| 10/15/2023 19:24:28 - INFO - app - STEP = 1796 | |
| 10/15/2023 19:24:29 - INFO - app - STEP = 1797 | |
| 10/15/2023 19:24:31 - INFO - app - STEP = 1798 | |
| 10/15/2023 19:24:32 - INFO - app - STEP = 1799 | |
| 10/15/2023 19:24:34 - INFO - app - STEP = 1800 | |
| 10/15/2023 19:24:34 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-1800 | |
| 10/15/2023 19:24:34 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-1800/optimizer.bin | |
| 10/15/2023 19:24:34 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-1800/scheduler.bin | |
| 10/15/2023 19:24:34 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-1800/random_states_0.pkl | |
| 10/15/2023 19:24:34 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-1800 | |
| 10/15/2023 19:24:35 - INFO - app - STEP = 1801 | |
| 10/15/2023 19:24:36 - INFO - app - STEP = 1802 | |
| 10/15/2023 19:24:37 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:25:03 - INFO - app - STEP = 1803 | |
| 10/15/2023 19:25:04 - INFO - app - STEP = 1804 | |
| 10/15/2023 19:25:06 - INFO - app - STEP = 1805 | |
| 10/15/2023 19:25:07 - INFO - app - STEP = 1806 | |
| 10/15/2023 19:25:08 - INFO - app - STEP = 1807 | |
| 10/15/2023 19:25:10 - INFO - app - STEP = 1808 | |
| 10/15/2023 19:25:11 - INFO - app - STEP = 1809 | |
| 10/15/2023 19:25:12 - INFO - app - STEP = 1810 | |
| 10/15/2023 19:25:13 - INFO - app - STEP = 1811 | |
| 10/15/2023 19:25:15 - INFO - app - STEP = 1812 | |
| 10/15/2023 19:25:16 - INFO - app - STEP = 1813 | |
| 10/15/2023 19:25:17 - INFO - app - STEP = 1814 | |
| 10/15/2023 19:25:19 - INFO - app - STEP = 1815 | |
| 10/15/2023 19:25:20 - INFO - app - STEP = 1816 | |
| 10/15/2023 19:25:21 - INFO - app - STEP = 1817 | |
| 10/15/2023 19:25:23 - INFO - app - STEP = 1818 | |
| 10/15/2023 19:25:24 - INFO - app - STEP = 1819 | |
| 10/15/2023 19:25:25 - INFO - app - STEP = 1820 | |
| 10/15/2023 19:25:26 - INFO - app - STEP = 1821 | |
| 10/15/2023 19:25:28 - INFO - app - STEP = 1822 | |
| 10/15/2023 19:25:29 - INFO - app - STEP = 1823 | |
| 10/15/2023 19:25:30 - INFO - app - STEP = 1824 | |
| 10/15/2023 19:25:32 - INFO - app - STEP = 1825 | |
| 10/15/2023 19:25:33 - INFO - app - STEP = 1826 | |
| 10/15/2023 19:25:34 - INFO - app - STEP = 1827 | |
| 10/15/2023 19:25:36 - INFO - app - STEP = 1828 | |
| 10/15/2023 19:25:37 - INFO - app - STEP = 1829 | |
| 10/15/2023 19:25:38 - INFO - app - STEP = 1830 | |
| 10/15/2023 19:25:39 - INFO - app - STEP = 1831 | |
| 10/15/2023 19:25:41 - INFO - app - STEP = 1832 | |
| 10/15/2023 19:25:42 - INFO - app - STEP = 1833 | |
| 10/15/2023 19:25:43 - INFO - app - STEP = 1834 | |
| 10/15/2023 19:25:45 - INFO - app - STEP = 1835 | |
| 10/15/2023 19:25:46 - INFO - app - STEP = 1836 | |
| 10/15/2023 19:25:47 - INFO - app - STEP = 1837 | |
| 10/15/2023 19:25:49 - INFO - app - STEP = 1838 | |
| 10/15/2023 19:25:50 - INFO - app - STEP = 1839 | |
| 10/15/2023 19:25:51 - INFO - app - STEP = 1840 | |
| 10/15/2023 19:25:52 - INFO - app - STEP = 1841 | |
| 10/15/2023 19:25:54 - INFO - app - STEP = 1842 | |
| 10/15/2023 19:25:55 - INFO - app - STEP = 1843 | |
| 10/15/2023 19:25:56 - INFO - app - STEP = 1844 | |
| 10/15/2023 19:25:57 - INFO - app - STEP = 1845 | |
| 10/15/2023 19:25:59 - INFO - app - STEP = 1846 | |
| 10/15/2023 19:26:00 - INFO - app - STEP = 1847 | |
| 10/15/2023 19:26:01 - INFO - app - STEP = 1848 | |
| 10/15/2023 19:26:03 - INFO - app - STEP = 1849 | |
| 10/15/2023 19:26:04 - INFO - app - STEP = 1850 | |
| 10/15/2023 19:26:05 - INFO - app - STEP = 1851 | |
| 10/15/2023 19:26:07 - INFO - app - STEP = 1852 | |
| 10/15/2023 19:26:08 - INFO - app - STEP = 1853 | |
| 10/15/2023 19:26:09 - INFO - app - STEP = 1854 | |
| 10/15/2023 19:26:10 - INFO - app - STEP = 1855 | |
| 10/15/2023 19:26:12 - INFO - app - STEP = 1856 | |
| 10/15/2023 19:26:13 - INFO - app - STEP = 1857 | |
| 10/15/2023 19:26:15 - INFO - app - STEP = 1858 | |
| 10/15/2023 19:26:16 - INFO - app - STEP = 1859 | |
| 10/15/2023 19:26:17 - INFO - app - STEP = 1860 | |
| 10/15/2023 19:26:18 - INFO - app - STEP = 1861 | |
| 10/15/2023 19:26:20 - INFO - app - STEP = 1862 | |
| 10/15/2023 19:26:21 - INFO - app - STEP = 1863 | |
| 10/15/2023 19:26:22 - INFO - app - STEP = 1864 | |
| 10/15/2023 19:26:23 - INFO - app - STEP = 1865 | |
| 10/15/2023 19:26:25 - INFO - app - STEP = 1866 | |
| 10/15/2023 19:26:26 - INFO - app - STEP = 1867 | |
| 10/15/2023 19:26:27 - INFO - app - STEP = 1868 | |
| 10/15/2023 19:26:29 - INFO - app - STEP = 1869 | |
| 10/15/2023 19:26:30 - INFO - app - STEP = 1870 | |
| 10/15/2023 19:26:31 - INFO - app - STEP = 1871 | |
| 10/15/2023 19:26:33 - INFO - app - STEP = 1872 | |
| 10/15/2023 19:26:34 - INFO - app - STEP = 1873 | |
| 10/15/2023 19:26:35 - INFO - app - STEP = 1874 | |
| 10/15/2023 19:26:36 - INFO - app - STEP = 1875 | |
| 10/15/2023 19:26:38 - INFO - app - STEP = 1876 | |
| 10/15/2023 19:26:39 - INFO - app - STEP = 1877 | |
| 10/15/2023 19:26:40 - INFO - app - STEP = 1878 | |
| 10/15/2023 19:26:41 - INFO - app - STEP = 1879 | |
| 10/15/2023 19:26:43 - INFO - app - STEP = 1880 | |
| 10/15/2023 19:26:44 - INFO - app - STEP = 1881 | |
| 10/15/2023 19:26:45 - INFO - app - STEP = 1882 | |
| 10/15/2023 19:26:47 - INFO - app - STEP = 1883 | |
| 10/15/2023 19:26:48 - INFO - app - STEP = 1884 | |
| 10/15/2023 19:26:49 - INFO - app - STEP = 1885 | |
| 10/15/2023 19:26:51 - INFO - app - STEP = 1886 | |
| 10/15/2023 19:26:52 - INFO - app - STEP = 1887 | |
| 10/15/2023 19:26:53 - INFO - app - STEP = 1888 | |
| 10/15/2023 19:26:54 - INFO - app - STEP = 1889 | |
| 10/15/2023 19:26:56 - INFO - app - STEP = 1890 | |
| 10/15/2023 19:26:57 - INFO - app - STEP = 1891 | |
| 10/15/2023 19:26:58 - INFO - app - STEP = 1892 | |
| 10/15/2023 19:27:00 - INFO - app - STEP = 1893 | |
| 10/15/2023 19:27:01 - INFO - app - STEP = 1894 | |
| 10/15/2023 19:27:02 - INFO - app - STEP = 1895 | |
| 10/15/2023 19:27:04 - INFO - app - STEP = 1896 | |
| 10/15/2023 19:27:05 - INFO - app - STEP = 1897 | |
| 10/15/2023 19:27:06 - INFO - app - STEP = 1898 | |
| 10/15/2023 19:27:07 - INFO - app - STEP = 1899 | |
| 10/15/2023 19:27:09 - INFO - app - STEP = 1900 | |
| 10/15/2023 19:27:10 - INFO - app - STEP = 1901 | |
| 10/15/2023 19:27:11 - INFO - app - STEP = 1902 | |
| 10/15/2023 19:27:12 - INFO - app - Running validation... | |
| Generating 4 images with prompt: A photo of sks woman smiling at the camera. | |
| 10/15/2023 19:27:38 - INFO - app - STEP = 1903 | |
| 10/15/2023 19:27:39 - INFO - app - STEP = 1904 | |
| 10/15/2023 19:27:41 - INFO - app - STEP = 1905 | |
| 10/15/2023 19:27:42 - INFO - app - STEP = 1906 | |
| 10/15/2023 19:27:43 - INFO - app - STEP = 1907 | |
| 10/15/2023 19:27:45 - INFO - app - STEP = 1908 | |
| 10/15/2023 19:27:46 - INFO - app - STEP = 1909 | |
| 10/15/2023 19:27:47 - INFO - app - STEP = 1910 | |
| 10/15/2023 19:27:48 - INFO - app - STEP = 1911 | |
| 10/15/2023 19:27:50 - INFO - app - STEP = 1912 | |
| 10/15/2023 19:27:51 - INFO - app - STEP = 1913 | |
| 10/15/2023 19:27:52 - INFO - app - STEP = 1914 | |
| 10/15/2023 19:27:54 - INFO - app - STEP = 1915 | |
| 10/15/2023 19:27:55 - INFO - app - STEP = 1916 | |
| 10/15/2023 19:27:56 - INFO - app - STEP = 1917 | |
| 10/15/2023 19:27:58 - INFO - app - STEP = 1918 | |
| 10/15/2023 19:27:59 - INFO - app - STEP = 1919 | |
| 10/15/2023 19:28:00 - INFO - app - STEP = 1920 | |
| 10/15/2023 19:28:01 - INFO - app - STEP = 1921 | |
| 10/15/2023 19:28:03 - INFO - app - STEP = 1922 | |
| 10/15/2023 19:28:04 - INFO - app - STEP = 1923 | |
| 10/15/2023 19:28:05 - INFO - app - STEP = 1924 | |
| 10/15/2023 19:28:06 - INFO - app - STEP = 1925 | |
| 10/15/2023 19:28:08 - INFO - app - STEP = 1926 | |
| 10/15/2023 19:28:09 - INFO - app - STEP = 1927 | |
| 10/15/2023 19:28:10 - INFO - app - STEP = 1928 | |
| 10/15/2023 19:28:12 - INFO - app - STEP = 1929 | |
| 10/15/2023 19:28:13 - INFO - app - STEP = 1930 | |
| 10/15/2023 19:28:14 - INFO - app - STEP = 1931 | |
| 10/15/2023 19:28:16 - INFO - app - STEP = 1932 | |
| 10/15/2023 19:28:17 - INFO - app - STEP = 1933 | |
| 10/15/2023 19:28:18 - INFO - app - STEP = 1934 | |
| 10/15/2023 19:28:19 - INFO - app - STEP = 1935 | |
| 10/15/2023 19:28:21 - INFO - app - STEP = 1936 | |
| 10/15/2023 19:28:22 - INFO - app - STEP = 1937 | |
| 10/15/2023 19:28:23 - INFO - app - STEP = 1938 | |
| 10/15/2023 19:28:25 - INFO - app - STEP = 1939 | |
| 10/15/2023 19:28:26 - INFO - app - STEP = 1940 | |
| 10/15/2023 19:28:27 - INFO - app - STEP = 1941 | |
| 10/15/2023 19:28:29 - INFO - app - STEP = 1942 | |
| 10/15/2023 19:28:30 - INFO - app - STEP = 1943 | |
| 10/15/2023 19:28:31 - INFO - app - STEP = 1944 | |
| 10/15/2023 19:28:32 - INFO - app - STEP = 1945 | |
| 10/15/2023 19:28:34 - INFO - app - STEP = 1946 | |
| 10/15/2023 19:28:35 - INFO - app - STEP = 1947 | |
| 10/15/2023 19:28:36 - INFO - app - STEP = 1948 | |
| 10/15/2023 19:28:37 - INFO - app - STEP = 1949 | |
| 10/15/2023 19:28:39 - INFO - app - STEP = 1950 | |
| 10/15/2023 19:28:40 - INFO - app - STEP = 1951 | |
| 10/15/2023 19:28:41 - INFO - app - STEP = 1952 | |
| 10/15/2023 19:28:43 - INFO - app - STEP = 1953 | |
| 10/15/2023 19:28:44 - INFO - app - STEP = 1954 | |
| 10/15/2023 19:28:45 - INFO - app - STEP = 1955 | |
| 10/15/2023 19:28:47 - INFO - app - STEP = 1956 | |
| 10/15/2023 19:28:48 - INFO - app - STEP = 1957 | |
| 10/15/2023 19:28:49 - INFO - app - STEP = 1958 | |
| 10/15/2023 19:28:50 - INFO - app - STEP = 1959 | |
| 10/15/2023 19:28:52 - INFO - app - STEP = 1960 | |
| 10/15/2023 19:28:53 - INFO - app - STEP = 1961 | |
| 10/15/2023 19:28:54 - INFO - app - STEP = 1962 | |
| 10/15/2023 19:28:56 - INFO - app - STEP = 1963 | |
| 10/15/2023 19:28:57 - INFO - app - STEP = 1964 | |
| 10/15/2023 19:28:58 - INFO - app - STEP = 1965 | |
| 10/15/2023 19:29:00 - INFO - app - STEP = 1966 | |
| 10/15/2023 19:29:01 - INFO - app - STEP = 1967 | |
| 10/15/2023 19:29:02 - INFO - app - STEP = 1968 | |
| 10/15/2023 19:29:04 - INFO - app - STEP = 1969 | |
| 10/15/2023 19:29:05 - INFO - app - STEP = 1970 | |
| 10/15/2023 19:29:06 - INFO - app - STEP = 1971 | |
| 10/15/2023 19:29:08 - INFO - app - STEP = 1972 | |
| 10/15/2023 19:29:09 - INFO - app - STEP = 1973 | |
| 10/15/2023 19:29:10 - INFO - app - STEP = 1974 | |
| 10/15/2023 19:29:11 - INFO - app - STEP = 1975 | |
| 10/15/2023 19:29:13 - INFO - app - STEP = 1976 | |
| 10/15/2023 19:29:14 - INFO - app - STEP = 1977 | |
| 10/15/2023 19:29:15 - INFO - app - STEP = 1978 | |
| 10/15/2023 19:29:17 - INFO - app - STEP = 1979 | |
| 10/15/2023 19:29:18 - INFO - app - STEP = 1980 | |
| 10/15/2023 19:29:19 - INFO - app - STEP = 1981 | |
| 10/15/2023 19:29:21 - INFO - app - STEP = 1982 | |
| 10/15/2023 19:29:22 - INFO - app - STEP = 1983 | |
| 10/15/2023 19:29:23 - INFO - app - STEP = 1984 | |
| 10/15/2023 19:29:24 - INFO - app - STEP = 1985 | |
| 10/15/2023 19:29:26 - INFO - app - STEP = 1986 | |
| 10/15/2023 19:29:27 - INFO - app - STEP = 1987 | |
| 10/15/2023 19:29:28 - INFO - app - STEP = 1988 | |
| 10/15/2023 19:29:30 - INFO - app - STEP = 1989 | |
| 10/15/2023 19:29:31 - INFO - app - STEP = 1990 | |
| 10/15/2023 19:29:32 - INFO - app - STEP = 1991 | |
| 10/15/2023 19:29:34 - INFO - app - STEP = 1992 | |
| 10/15/2023 19:29:35 - INFO - app - STEP = 1993 | |
| 10/15/2023 19:29:36 - INFO - app - STEP = 1994 | |
| 10/15/2023 19:29:37 - INFO - app - STEP = 1995 | |
| 10/15/2023 19:29:39 - INFO - app - STEP = 1996 | |
| 10/15/2023 19:29:40 - INFO - app - STEP = 1997 | |
| 10/15/2023 19:29:41 - INFO - app - STEP = 1998 | |
| 10/15/2023 19:29:43 - INFO - app - STEP = 1999 | |
| 10/15/2023 19:29:44 - INFO - app - STEP = 2000 | |
| 10/15/2023 19:29:45 - INFO - accelerate.accelerator - Saving current state to models/SD-Jenniger-Test3/checkpoint-2000 | |
| 10/15/2023 19:29:45 - INFO - accelerate.checkpointing - Optimizer state saved in models/SD-Jenniger-Test3/checkpoint-2000/optimizer.bin | |
| 10/15/2023 19:29:45 - INFO - accelerate.checkpointing - Scheduler state saved in models/SD-Jenniger-Test3/checkpoint-2000/scheduler.bin | |
| 10/15/2023 19:29:45 - INFO - accelerate.checkpointing - Random states saved in models/SD-Jenniger-Test3/checkpoint-2000/random_states_0.pkl | |
| 10/15/2023 19:29:45 - INFO - app - Saved state to models/SD-Jenniger-Test3/checkpoint-2000 | |