Newspapers
Collection
Collection of models finetuned/adapted on 19th-century British newspaper collections • 6 items • Updated
NewsBERT 1890–1900 LoRA adapter (1 epoch)
A LoRA adapter for TextMachineProject/NewsBERT_1800-1920, fine-tuned for one epoch on newspaper text (1890–1900) from the Heritage Made Digital (HMD14) and Living with Machines (LwM) collections.
Training details
- Period: 1890–1900
- Base model:
TextMachineProject/NewsBERT_1800-1920 - Method: LoRA (PEFT), target modules:
query,value,word_embeddings - LoRA rank: 16, alpha: 32, dropout: 0.05
- Task: Masked Language Modelling (15% masking probability)
- Sequence length: 128 tokens (sliding window, stride 96)
- Epochs: 1
- Batch size: 256
Usage
from transformers import AutoTokenizer, AutoModelForMaskedLM
from peft import PeftModel
base = AutoModelForMaskedLM.from_pretrained("TextMachineProject/NewsBERT_1800-1920")
tokenizer = AutoTokenizer.from_pretrained("TextMachineProject/NewsBERT_1800-1920")
model = PeftModel.from_pretrained(base, "TextMachineProject/NewsBERT_1890_1900_lora_1epoch")
Notes
This is a 1-epoch checkpoint uploaded for evaluation purposes. Further training is ongoing; updated adapters will be released separately.
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for TextMachineProject/NewsBERT_1890_1900_lora_1epoch
Base model
TextMachineProject/NewsBERT_1800-1920