Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini
Authors: name: Madhuri Shanbhogue, name: Zhe Li, name: Shanfeng Zhang, name: Gustavo Hernández Ábrego, name: Shih-Cheng Huang, name: Aashi Jain, name: Daniel Salz, name: Sonam Goenka, name: Chaitra Hegde, name: Ji Ma, name: Feiyang Chen, name: Jiaxing Wu, name: Tanmaya Dabral, name: Babak Samari, name: Kevin Poulet, name: Daniel Cer, name: Kaifeng Chen, name: Paul Suganathan, name: Hui Hui, name: Jovan Andonov, name: Philippe Schlattner, name: Jay Han, name: Iftekhar Naim, name: Wing Lowe, name: Vladimir Pchelin, name: Albert Yang, name: Yi-Ting Chen, name: Zhongli Ding, name: Grace Zhang, name: Georg Heigold, name: Yichang Chen, name: Antoine Reveillon, name: Brendan Mccloskey, name: Wenlei Zhou, name: Dahun Kim, name: Rui Meng, name: Emma Wang, name: Jack Zheng, name: Halley Fede, name: Zhen Yang, name: Keegan Mosley, name: Brian Potetz, name: Sahil Dua, name: Henrique Schechter Vera, name: Shen Gao, name: Hesen Zhang, name: Andreas Hess, name: Hengxuan Ying, name: Alberto Montes, name: Karan Gill, name: Min Choi, name: Sebastian Russo, name: Anja Hauth, name: Jinhyuk Lee, name: Michael Boratko, name: Megan Barnes, name: Vikram Rao, name: Claudiu Musat, name: Cyril Allauzen, name: Ehsan Variani, name: Shankar Kumar, name: Tom Bagby, name: Junyi Jiao, name: Yang Gu, name: Tengxin Li, name: Ayush Agrawal, name: Roberto Santana, name: Dev Nath, name: Stephen Karukas, name: Shuoxuan Han, name: Lucia Loher, name: Alice Twu, name: Nidhi Vyas, name: Siddharth Bhai, name: Frank Palma Gomez, name: Wangyuan Zhang, name: Chaoren Liu, name: Jizheng Yang, name: Steve Qiu, name: Shijie Zhang, name: Sujay Kulkarni, name: Sascha Rothe, name: Sean Nakamoto, name: Raphael Hoffmann, name: Zach Gleicher, name: Yunhsuan Sung, name: Qin Yin, name: Tom Duerig, name: Mojtaba Seyedhosseini
Organization: Google

We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage the multimodal capabilities of Gemini to produce embeddings for arbitrary combinations of interleaved inputs across all these modalities that generalize well across a wide variety of tasks. Applying large-scale contrastive learning in a multi-task multi-stage training setup, we achieve state-of-the-art performance on key embedding benchmarks including unimodal, cross-modal, and multimodal retrieval spanning a diverse set of tasks. We show that our embedding model demonstrates strong performance (with a score of 62.9 R@1 on MSCOCO, 68.8 NDCG@10 on Vatex, 69.9 on MTEB multilingual and 84.0 on MTEB Code) across a variety of tasks surpassing the performance of specialized models. These unified capabilities make Gemini Embedding 2 a promising candidate for downstream use cases such as RAG, recommendation and search. Furthermore, its robust zero-shot performance across distinct fields - from astronomy and bioscience to fine arts and the culinary arts - establishes it as a highly reliable, out-of-the-box representation even for specialized domains.