BioLoom

BioLoom is a biomedical vision-language model for image-text representation learning. This repository provides two pretrained checkpoints with a ViT-L/14 vision encoder and a BioClinical ModernBERT text encoder. The image input size is 224 pixels and the text context length is 512 tokens.

Released checkpoints

• BioLoom (Scratch): random vision initialization, trained for 10 epochs. Download random_vision_10epoch.pt.
• BioLoom (OpenCLIP initialization): pretrained vision initialization, trained for 10 epochs. Download openclip_init_10epoch_best.pt.

The architecture configuration, checkpoint loader, and ModernBERT compatibility helper are included. Repository access is gated; request access and authenticate with Hugging Face before downloading.

Installation

Download the repository's Python files, configuration and requirements file into the same working directory, then run:

pip install -r requirements.txt
hf auth login

Load a model

python load_model.py --checkpoint random_vision_10epoch.pt --device cuda

For the pretrained vision variant:

python load_model.py --checkpoint openclip_init_10epoch_best.pt --device cuda

The Python interface returns the model, image preprocessing transform, and tokenizer:

from load_model import load_model

model, preprocess, tokenizer = load_model(
    "random_vision_10epoch.pt", device="cuda"
)

To use a checkpoint already stored locally, set BIOLOOM_CHECKPOINT to its file path. Model configuration and tokenizer assets may still require downloading.

Repository contents

• checkpoints/: the two released model weights.
• MODEL_INFO.json: architecture and checkpoint metadata.
• BioClinicalModernBERT-large-ViT-L-14-pretrained.json: OpenCLIP architecture configuration.
• load_model.py: checkpoint loader.
• openclip_compat.py: ModernBERT compatibility helper.
• requirements.txt: inference dependencies.

The model produces image and text embeddings that can be compared using cosine similarity for retrieval or zero-shot classification.