Semantic Token Reweighting for Interpretable and Controllable Text Embeddings in CLIP (2024)

Abstract: A text encoder within Vision-Language Models (VLMs) like CLIP plays a crucial role in translating textual input into an embedding space shared with images, thereby facilitating the interpretative analysis of vision tasks through natural language. Despite the varying significance of different textual elements within a sentence depending on the context, efforts to account for variation of importance in constructing text embeddings have been lacking. We propose a framework of Semantic Token Reweighting to build Interpretable text embeddings (SToRI), which incorporates controllability as well. SToRI refines the text encoding process in CLIP by differentially weighting semantic elements based on contextual importance, enabling finer control over emphasis responsive to data-driven insights and user preferences. The efficacy of SToRI is demonstrated through comprehensive experiments on few-shot image classification and image retrieval tailored to user preferences.

## introduction as artificial intelligence (ai) systems based on deep learning models grow in application in our daily lives, their black box nature raises issues of transparency, resulting in a demand for enhanced interpretability to promote trust in ai systems @xcite . consequently, research efforts have been focused on making the systems' decision-making processes more humanunderstandable through various explanatory methods @xcite @xcite . among the various forms of explanation, natural language has emerged as an excellent medium due to its humanfriendly nature and adeptness in managing highlevel abstractions @xcite . these advantages have led to a growing interest in leveraging natural language for interpreting vision tasks @xcite . to facilitate the use of natural language in vision tasks, vision-language models (vlms) like clip @xcite are commonly deployed to bridge visual information and its linguistic interpretation @xcite @xcite . clip consists of two encoders that translate images and texts into embeddings that coexist in a shared space, enabling vision tasks to be conducted and understood through natural language. natural language sentences often carry multiple implications, with varying levels of significance that can change based on the desired outcome, even if the text remains unchanged. selectively emphasizing certain information relevant to a task can aid in conducting and understanding the task. for instance, when differentiating given images of a 'great grey owl' from other groups using the description 'a large owl with big yellow eyes', emphasis on 'eyes' may better represent the group of images, indicating that 'eyes' is significant (see examples of image classification in figure 1 ). similarly, when searching for images using the query 'a castle surrounded by trees,' the preference on 'trees' relative to 'a castle' could differ based on user intent, and retrieval reflecting this can yield the desired search results (see examples of retrieved images in figure 1 ). while there have been attempts to modulate focus in image and text generation @xcite @xcite , fine-tuning the importance of specific text elements in clip's text embeddings remains relatively unexplored. this paper endeavors to create text embeddings that can incorporate a varying controlled importance of each semantic element within a sentence, thereby enhancing the representativeness of text embedding for images in interpretable way. ## preliminary: text embeddings in clip the text encoder of clip @xcite , which utilizes a transformer-based architecture, transforms a given text prompt into a single vector through the following process. initially, a given text prompt is converted into a sequence of text tokens {x i } n @xmath0, where n represents the number of the text tokens. tokens indicating the start and end, [sos] and [eos] tokens, are appended at the beginning and the end of the sequence of tokens, resulting in the extended series {x i } n +1 @xmath1, with x 0 and x n +1 representing the [sos] and [eos], respectively. each text token is then converted into an embedded input token, and positional embedding is added, resulting in the input embedding for the first transformer block {z 0 i } n +1 @xmath2. for the l-th block of the encoder, the input tokens can be represented as the output tokens from the l-th block is given by: scaling and masking operations are omitted for simplicity. through the attention mechanism, tokens influence each other, and the values of a represent the extent to which they influence one another @xcite . in general, the final output text embedding of the [eos] token encapsulates the full semantic meaning of the text prompt. this embedding is compared with image embeddings to assess the degree of correspondence with images once it has been projected into a multimodal embedding space. a pre-trained clip model is commonly employed for image classification, where given an image, it computes similarity scores with class names, which become logits. to adapt the model to a specific dataset, fine-tuning is performed by minimizing the cross-entropy loss as follows: ## method we propose stori, a novel framework that encodes a given text prompt into a single text embedding vector using clip by varying the importance of different textual elements through data-driven and user-driven controls. in section 3.1, we elaborate on semantic token reweighting, which involves modifying the attention given to individual tokens within the text encoding process based on their respective weights. in section 3.2, we present two methods for determining these weights. ## experiments we evaluate stori on two representative vision tasks, few-shot image classification and preferencebased image retrieval. in few-shot image classification, weights are determined via data-driven control and provide interpretation on the trained classifier. evaluation on preference-based image retrieval demonstrates controllability of stori via user-driven control. ## related works vlms and interpretability in recent vision tasks, interpretative analysis in natural language becomes popular rather than relying solely on visual form. vlms like clip have commonly been employed to connect the image and text feature spaces for explanations. @xcite utilized clip to derive concept activation vector @xcite in vision model, while @xcite and @xcite leveraged clip to identify whether concepts defined in text are present in images. @xcite and @xcite crafted text prompts to explain image classes using large language models (llms), applying them for zero-shot classification with clip. beyond simply utilizing clip's shared embedding space, several works have focused on enhancing the interpretability of this space. @xcite introduced the stair model, which improves interpretability by fine-tuning clip to generate sparse, more interpretable representations. @xcite , on the other hand, proposed decomposing image embeddings into conceptual text embeddings, combining concepts within the embedding space for clearer explanations. in contrast to these methods, we ensure interpretability by manipulating token weighting within a given description's context, without the need for model fine-tuning or embedding decomposition. this approach allows for practical applications in controlling interpretation, such as adjusting token importance to reflect user preferences or contextual emphasis, while also enabling detailed analysis of interpretability. few-shot image classification clip exhibits promising performance in image recognition tasks, leading to the development of various few-shot learning approaches. coop @xcite and cocoop @xcite are representative methods based on prompt tuning. tip-adapter @xcite integrates an extra adapter unit following the encoders. taskres @xcite involves training task-specific residual text embeddings for each category. while these approaches incorporate extra trainable parameters outside an interpretable framework and thus do not guarantee interpretability, our framework enables the training of classifiers while ensuring interpretability. ## conclusion we propose stori, a framework that builds interpretable text embeddings by reweighting semantic tokens in clip. this approach innovatively enhances the explanatory power of natural language in vision tasks. our control strategies enable tuning of text embeddings for classification and retrieval while maintaining interpretability. stori can be easily applied to any model based on attention mechanisms and has potential scalability across various vision tasks. additionally, our method of reweighting text prompt tokens during text encoding can similarly be applied to image encoding. this approach can allow for the emphasis of specific image regions. the extension to multi-modal tasks using diverse vlms remains a topic for future work.