The Slerp-based Zero-Shot Composed Image Retrieval (ZS-CIR) approach can be extended to handle various composed retrieval scenarios by adapting the interpolation technique to accommodate different types of queries and retrieval galleries.
Queries with Both Image and Text: To handle queries that consist of both image and text, the Slerp method can be modified to incorporate both modalities effectively. By adjusting the interpolation process to consider the joint representation of image and text, the Slerp-based approach can generate composed embeddings that capture the combined information from both modalities. This would involve modifying the interpolation formula to account for the dual input nature of the query.
Retrieval Galleries with Both Image and Text Samples: When dealing with retrieval galleries that contain both image and text samples, the Slerp-based approach can be enhanced to compare the query's composed representation with a diverse set of gallery items. By extending the interpolation process to handle multiple modalities in the gallery, the method can effectively match the query's combined image-text representation with the diverse content in the gallery. This extension would involve adapting the similarity computation step to consider the mixed nature of the gallery items.
By customizing the Slerp-based approach to address these different composed retrieval scenarios, it can offer a versatile solution for handling a wide range of queries and retrieval galleries in vision-language tasks.