Efficient Code Embeddings from Code Generation Models
Authors: name: Daria Kryvosheieva; name: Saba Sturua; name: Michael Günther; name: Scott Martens; name: Han Xiao
Published: 2025-08-29 01:18:15

Abstract: jina-code-embeddings is a novel code embedding model suite designed to
retrieve code from natural language queries, perform technical
question-answering, and identify semantically similar code snippets across
programming languages. It makes innovative use of an autoregressive backbone
pre-trained on both text and code, generating embeddings via last-token
pooling. We outline the training recipe and demonstrate state-of-the-art
performance despite the relatively small size of the models, validating this
approach to code embedding model construction.