Simple controlnet model made as part of the HF JaX/Diffusers community sprint.
These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with pose conditioning generated using the animalpose model of OpenPifPaf.
Some example images can be found in the following
Whilst not the dataset used for this model, a smaller dataset with the same format for conditioning images can be found at https://huggingface.co/datasets/JFoz/dog-poses-controlnet-dataset
The dataset was generated using the code at https://github.com/jfozard/animalpose/tree/f1be80ed29886a1314054b87f2a8944ea98997ac
This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf https://openpifpaf.github.io/intro.html . Skeleton colouring is as shown in the dataset. See also https://huggingface.co/JFoz/dog-pose
This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf https://openpifpaf.github.io/intro.html. Skeleton colouring is as shown in the dataset. See also https://huggingface.co/JFoz/dog-pose
- Developed by: John Fozard
- Model type: Conditional image generation
- Language(s) (NLP): en
- License: openrail
- Parent Model: https://huggingface.co/runwayml/stable-diffusion-v1-5
- Resources for more information:
Supply a suitable, potentially incomplete pose along with a relevant text prompt
Generating images of non-animals. We advise retaining the stable diffusion safety filter when using this model.
The model is trained on a relatively small dataset, and may be overfit to those images.
Maintain careful supervision of model inputs and outputs.
Trained on a subset of Laion-5B using clip retrieval with the prompts "a photo of a (dog/cat) (standing/walking)"
Images were rescaled to 512 along their short edge and centrally cropped. The OpenPifPaf pose-detection model was used to extract poses, which were used to generate conditioning images.
Flax stable diffusion controlnet pipeline
- Downloads last month