klpeng/uce-gemma-80k

UCE Gemma

Convert text into scores for 80,513 named concepts.

Install

Replace YOUR_USERNAME with the Hugging Face repository owner:

pip install -U huggingface_hub
hf download YOUR_USERNAME/uce-gemma-80k --local-dir uce-gemma-80k
cd uce-gemma-80k
pip install -r requirements.txt

Run

Run Python from the downloaded directory:

from uce_gemma import UCEGemma

model = UCEGemma.from_pretrained(".", device="cuda")
scores = model.encode("Djokovic overwhelms Nadal for Wimbledon Title")

for concept in model.explain(scores[0], top_n=20):
    print(concept["concept_id"], concept["label"], concept["activation"])

Use device="cpu" without a CUDA GPU. Pass a list of strings to encode multiple texts. Inputs can contain up to 8,192 tokens, including the document prompt.

scores is a sparse matrix of shape (number_of_texts, 80513), retaining the 128 highest-scoring concepts per text. Values are sigmoid scores, not calibrated probabilities.

Baseline prevalences

prevalences.csv and prevalences.npz contain the same baseline statistics for all 80,513 concepts, joined to labels.csv and output columns by concept_id. They include counts, prevalence fractions, and one-sided 99% Clopper-Pearson upper bounds at activation thresholds >0.05 and >0.1.

These files are copied unchanged from Gemini UCD. They describe 89,827,558 stored Gemini corpus rows, including duplicates. They were computed using the historical 131,072-feature Gemini encoder's global top-128 selection, then restricted and remapped to these 80,513 concept IDs. They are historical Gemini teacher baselines, not prevalences measured from UCE Gemma outputs; the thresholds apply to the historical Gemini activations, not Gemma sigmoid scores. Use them as heuristic reference rates for concept enrichment.

prevalences.json records the original population, computation, and confidence bound assumptions. package_provenance.json records the source repository, pinned revision, and file hashes.

import csv

with open("prevalences.csv", newline="") as f:
    baselines = {int(row["concept_id"]): row for row in csv.DictReader(f)}

concept_id = 0
print(float(baselines[concept_id]["prevalence_gt_0p05"]))