Back to all models
Model card Files and versions Use in transformers
text-generation mask_token:
Query this model
🔥 This model is currently loaded and running on the Inference API. ⚠️ This model could not be loaded by the inference API. ⚠️ This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint  

⚡️ Upgrade your account to access the Inference API

Share Copied link to clipboard

Contributed by

Google AI company
6 team members · 115 models

Reformer Model trained on "Crime and Punishment"

Crime and Punishment is a novel written by Fyodor Dostoevsky and was translated into English.

Crime and Punishment training data was taken from gs://trax-ml/reformer/crime-and-punishment-2554.txt and contains roughly 0.5M tokens.

The ReformerLM model was trained in flax using colab notebook proposed by authors: and the weights were converted to Hugging Face's PyTorch ReformerLM model ReformerModelWithLMHead.

The model is a language model that operates on small sub-word units. Text can be generated as follows:

model = ReformerModelWithLMHead.from_pretrained("google/reformer-crime-and-punishment")
tok = ReformerTokenizer.from_pretrained("google/reformer-crime-and-punishment")
tok.decode(model.generate(tok.encode("A few months later", return_tensors="pt"), do_sample=True,temperature=0.7, max_length=100)[0])

# gives:'A few months later on was more than anything in the flat. 
# “I have already.” “That’s not my notion that he had forgotten him. 
# What does that matter? And why do you mean? It’s only another fellow,” he said as he went out, as though he want'