Text-to-Image
Diffusers
English
Edit model card

PhotoMaker Model Card

Introduction

Users can input one or a few face photos, along with a text prompt, to receive a customized photo or painting within seconds (no training required!). Additionally, this model can be adapted to any base model based on SDXL or used in conjunction with other LoRA modules.

Realistic results

image/jpeg

image/jpeg

Stylization results

image/jpeg

image/jpeg

More results can be found in our project page

Model Details

It mainly contains two parts corresponding to two keys in loaded state dict:

  1. id_encoder includes finetuned OpenCLIP-ViT-H-14 and a few fuse layers.

  2. lora_weights applies to all attention layers in the UNet, and the rank is set to 64.

Usage

You can directly download the model in this repository. You also can download the model in python script:

from huggingface_hub import hf_hub_download
photomaker_ckpt = hf_hub_download(repo_id="TencentARC/PhotoMaker", filename="photomaker-v1.bin", repo_type="model")

Then, please follow the instructions in our GitHub repository.

Limitations

  • The model's customization performance degrades on Asian male faces.
  • The model still struggles with accurately rendering human hands.

Bias

While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.

Citation

BibTeX:

@inproceedings{li2023photomaker,
  title={PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding},
  author={Li, Zhen and Cao, Mingdeng and Wang, Xintao and Qi, Zhongang and Cheng, Ming-Ming and Shan, Ying},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}
Downloads last month
43,516
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for TencentARC/PhotoMaker

Adapters
3 models

Spaces using TencentARC/PhotoMaker 100

Collection including TencentARC/PhotoMaker