|
--- |
|
language: zh |
|
tags: |
|
- roformer |
|
inference: false |
|
--- |
|
## 介绍 |
|
### tf版本 |
|
https://github.com/ZhuiyiTechnology/roformer |
|
|
|
### pytorch版本 |
|
https://github.com/JunnYu/RoFormer_pytorch |
|
|
|
## 使用 |
|
```python |
|
git clone https://github.com/JunnYu/RoFormer_pytorch |
|
cd RoFormer_pytorch |
|
import torch |
|
from model import RoFormerModel, RoFormerTokenizer |
|
tokenizer = RoFormerTokenizer.from_pretrained("junnyu/roformer_chinese_base") |
|
model = RoFormerModel.from_pretrained("junnyu/roformer_chinese_base") |
|
inputs = tokenizer(text, return_tensors="pt") |
|
with torch.no_grad(): |
|
outputs = model(**inputs).last_hidden_state |
|
print(outputs.shape) |
|
``` |
|
## 引用 |
|
|
|
Bibtex： |
|
|
|
```tex |
|
@techreport{zhuiyiroformer, |
|
title={RoFormer: Transformer with Rotary Position Embeddings - ZhuiyiAI}, |
|
author={Jianlin Su}, |
|
year={2021}, |
|
url="https://github.com/ZhuiyiTechnology/roformer", |
|
} |
|
``` |
