SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Point Cloud Conditioned Image Generation We take the Stable Diffusion v2.1 unCLIP model [56] for image generation and replace the CLIP image condition encoder with our OpenShape encoder to perform image generation conditioned on point clouds (and optionally text prompts). The unCLIP model takes CLIP ViT-L/14 embeddings without normalization as input. To match the embedding space, we trained a variant of OpenShape-PointBERT with CLIP ViT-L/14 embeddings. Additionally, we noticed a significant mismatching of scales ( $L_{2}$ -norm of embedding vectors) between ViT-L/14 image embeddings and OpenShape embeddings. To mitigate this issue, we perform a re-normalization on OpenShape embeddings to a  $L_{2}$ -norm of  $\frac{1}{2}\sqrt{768}$ , which is our observed mean  $L_{2}$ -norm of ViT-L/14 image embeddings. We use 50 diffusion steps. The guidance scale can be tuned freely.