[Bug and Question] download problem on iPad + converting inpainting model
Hello friends,
Since last update I have trouble downloading the models from the iOS app.
The download does not start at all (loading wheel) then a few minutes later this message pops :
I can't figure out the direct download method from url, I linked the chunked.zip without success
Also, I'm trying to convert custom inpainting but I got this error :
size mismatch for conv_in.weight: copying a param with shape torch.Size([320, 9, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).
Searching the internet lead to nothing related (some naming stuff), so I gave it a shot on sd-v1-5-inpainting and got same error.
Since we got a working inpainting model I guess there is some sprinkle of magic to add in the recipe ?
I have huge hopes in this app, send you love xD
Hi, Got your update from a a day ago, had same bug but different message
Thank you :)
@ZProphete hey! sorry for the late response, I think I fixed that problem on the iPad or at least the error should be much more informative. I'm also working on a huge update for the Converter that should allow converting much more models (safetensor models for example).
Now on the inpainting model conversion, that is tricky, since you seem really interested I can explain this a bit, as you can see the expected input when converting the Unet is 4, this means it's expecting an image (the current one being "diffused"), an inpainting model expects that plus the original image (another 4) and a mask (another 1) for a total of 9.
I could maybe add a toggle on the converter for "inpainting" models, or maybe I can just detect it during conversion, I have to check this again as I just did a quick hack for the 2 inpainting models I converted.
@GuiyeC
Hi, Thank you, the iPad download is fixed, it was a bit confusing cause there were no download progress.
I'm so glad about the future changes ! You are getting closer to perfection.
off topic :
Btw Is it OK to suggest features for the main app around here ? I don't want to pollute your pages.
For selfish me, after in-painting, I would love to be able to
- zoom/crop an image before generating variations (similar to auto1111 img2img "Only masked" option), that would boost my workflow !
- automation I/O (AppleScript?) so I can implement an editor plugin, some QuickAction or scripted keyboard shortcut that would "copy2Guernika/process/PasteFromGuernika" for editors that don't support plugins like Pixelmator Pro.
Love your work, the tech is going so fast, and you are ahead of other apps, you answer messages, you are a beast !
Waiting patiently for implementation of LoRA's in coreML by apple ;D
@ZProphete I just updated the GuernikaModelConverter and uploaded new versions for the main app with support for ControlNet, you may have more luck converting using this version or maybe I broke something 😅
As for the improvements, I should put some love into the editor, I had an idea of having some canvas like DALL-e has for inpainting/generation, allowing moving the output square, but that maybe is a little crazy.
What do you mean with your second suggestion? Could you elaborate a bit more?
@GuiyeC
Waow, amazing, the App Store didn't pop it up yet, but I'm excited.
- Yes please :D
- Sure, I'll try my best :
I often use Pixelmator Pro as editor when PS is overkill, I use AppleScript's from my top bar menu to get some actions done
It should be nice to be able to control both editor/Guernika this way
Here is a sample script that detects face from an image and exports it a square
(Exporting workspace/Slices is not supported so I used crop-export and undo)
I would drag the exported face.png back to Guernika manually, make some prompt, export, and back to editor.
It should be great, (but not top list regarding how quickly you have to implements crazy new stuffs) to make something like :
Edit :
- Starting from image editor, make a square selection, press the script, something : "Tell Guernika to grab that selection from file url or clipboard as img2img source"
- From Guernika, export last generation to clipboard or directly into image editor at position {X, Y} ...
There are documentations around "ScriptingBridge" for swift apps but I'm no swift dev.
It opens doors to make plugins using Guernika for PS or Pixelmator Pro.
I am CERTAINLY overthinking it : please make img2img source from clipboard (CMD+C from Image Editor, CMD+V on Guernika)
Love your work.
@ZProphete I took a quick look at this and I'm thinking would Shortcuts support be enough? At this point I think that may be easier to implement and probably more future proof, but maybe other editing apps would not work as nicely together, I'm genuinely curious about what you think as you really seem to know your stuff :)
@GuiyeC
Hi ! Of course, it can be very useful,
even if Shortcut lacks direct functionality on certain apps (in my case, no face detection, that's why I used AppleScript) there is certainly a workaround to get something working)
It would make Guernika closer to scripting like a popular web-ui :
I can imagine "X/Y plots", or word swap like "a beautiful cat | dog | penguin" done without cluttering your base code with a new syntax parser or w/e.
Yes I'll make a new demonic creature for my mother-in-law's contact picture, every, single, day :D