plesk8er/EmotionCLIP-V2-ONNX

EmotionCLIP-V2 ONNX for TouchDesigner

ONNX conversion of jiangchengchengNLP/EmotionCLIP-V2 (CLIP ViT-B/32 tuned on EmoSet with prefix, prompt and layer-norm tuning), packaged for the Convivial Studio EmotionCLIP component for TouchDesigner. The weights are unchanged; only the format differs.

File	What it is
emotionclip_v2_vision[_fp16].onnx	image encoder, image float32 [1,3,224,224] RGB 0..1 in, normalized 512-d embedding out. CLIP normalization is baked in.
emotionclip_v2_text[_fp16].onnx	text encoder with the learned emotion prompt, tokens int64 [1,57] in, normalized 512-d embedding out
emotionclip_v2_bank.json	precomputed embeddings for the 9 trained labels and a default word bank, plus logit_scale
bpe_simple_vocab_16e6.txt.gz	CLIP BPE vocabulary from OpenAI CLIP (MIT license)

Labels: amusement, anger, awe, contentment, disgust, excitement, fear, sadness, neutral. Prompt template: This picture conveys a sense of {}. Score: softmax(logit_scale * text_embeddings @ image_embedding).

The model reads the affective tone of whole images (scenes, photos, artworks). It was not trained specifically on facial expressions. See the original model card for training data, results and known limitations.

License and credit

Apache-2.0, same as the original model. Original model by jiangchengchengNLP. ONNX conversion by Convivial Studio. The CLIP tokenizer vocabulary is from OpenAI CLIP (MIT).