---
library_name: transformers
tags: []
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Lora weights after finetuning Switch-base-128 on WMT16 datasets.

# To use Lora weights

```python
from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

base_model = AutoModelForSeq2SeqLM.from_pretrained('google/switch-base-128')
lora_model = PeftModel.from_pretrained(base_model, 'marsggbo/wmt-switch-base-128-lora-ckpt35000')
merged_model = lora_model.merge_and_unload()
merged_model.save_pretrained('./switch-base-128-wmt', state_dict=merged_model.state_dict(), safe_serialization=False)
```

# To use merged model weights

```python
model = AutoModelForSeq2SeqLM.from_pretrained('./switch-base-128-wmt')
```