Run BART with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/bart_large_xsum

Paper: BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (arXiv:1910.13461) · HF Papers

BART is a denoising seq2seq transformer: a bidirectional encoder (like BERT) and an autoregressive decoder (like GPT) trained to reconstruct corrupted text. It excels at summarization, translation, and other text-to-text tasks. Byte-level BPE tokenizer (shared with RoBERTa); the decoder starts from </s>.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of facebook/bart-large-xsum for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a extreme summarization (XSum, one-sentence) checkpoint (BartConditionalGenerate). Other task heads load the shared backbone from this repo (start randomly initialized, ready for fine-tuning); fine-tuned task checkpoints load via the hf: prefix.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.bart import BartConditionalGenerate, BartTokenizer

model = BartConditionalGenerate.from_weights("zeromodels/bart_large_xsum")
tokenizer = BartTokenizer.from_weights("zeromodels/bart_large_xsum")

inputs = tokenizer('The tower is 324 metres tall, about the same height as an 81-storey building, and the tallest structure in Paris.')
ids = model.generate(
    inputs,
    [[model.decoder_start_token_id]],
    max_new_tokens=64,
    eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(ids[0], skip_special_tokens=True))

Load any BART variant the same way with from_weights("zeromodels/<variant>"):

Variant	Hub	Task
`bart_base`	`zeromodels/bart_base`	conditional generation (base seq2seq)
`bart_large`	`zeromodels/bart_large`	conditional generation (base seq2seq)
`bart_large_cnn`	`zeromodels/bart_large_cnn`	summarization (CNN / DailyMail)
`bart_large_xsum`	`zeromodels/bart_large_xsum`	extreme summarization (XSum, one-sentence)

Available classes

Load any of these from this repo with from_weights("zeromodels/bart_large_xsum") (or on the fly via the hf: prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a hf: fine-tune).

Class	Task
`BartModel`	Encoder-decoder backbone
`BartConditionalGenerate`	Conditional generation (summarization / seq2seq)
`BartSequenceClassify`	Sequence classification (e.g. NLI / zero-shot)
`BartQnA`	Extractive question answering

from zeromodels.models.bart import BartSequenceClassify
# zero-shot / NLI fine-tune loads on the fly via the hf: prefix
model = BartSequenceClassify.from_weights("hf:facebook/bart-large-mnli")

Tips

Set KERAS_BACKEND before importing Keras / zeromodels.
Prefer BartTokenizer.from_weights(...) so the byte-level BPE matches.
BART's decoder starts from </s> (decoder_start_token_id = 2); pass eos_token_id=tokenizer.eos_token_id to stop generation.
See the BART docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. BartConditionalGenerate.from_weights("hf:facebook/bart-large-xsum").