|
--- |
|
tags: |
|
- roberta |
|
- adapterhub:nli/multinli |
|
- adapter-transformers |
|
license: apache-2.0 |
|
language: |
|
- en |
|
library_name: adapter-transformers |
|
--- |
|
|
|
# Adapter `yoh/distilroberta-base-sept-adapter` for distilroberta-base |
|
|
|
An [adapter](https://adapterhub.ml) for the `distilroberta-base` model that was trained on the [AllNLI](https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/datasets/paraphrases/AllNLI.jsonl.gz), [Sentence compression](https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/datasets/paraphrases/sentence-compression.jsonl.gz) and [Stackexchange duplicate question](https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/datasets/paraphrases/stackexchange_duplicate_questions.jsonl.gz) datasets (see information [here](https://www.sbert.net/examples/training/paraphrases/README.html)). |
|
|
|
This adapter was created for usage with the **[adapter-transformers](https://github.com/Adapter-Hub/adapter-transformers)** library. See this [paper](https://arxiv.org/abs/2311.00408) and [repository](https://github.com/UKPLab/AdaSent/blob/main/README.md) for more information on the tasks. |
|
|
|
## Usage |
|
|
|
First, install `adapter-transformers` and `sentence-transformers`: |
|
|
|
``` |
|
pip install -U adapter-transformers sentence-transformers |
|
``` |
|
_Note: adapter-transformers is a fork of transformers that acts as a drop-in replacement with adapter support. [More](https://docs.adapterhub.ml/installation.html)_ |
|
|
|
Now, the adapter can be loaded and activated like this: |
|
|
|
```python |
|
from sentence_transformers import SentenceTransformer, models |
|
|
|
# Load pre-trained model |
|
word_embedding_model = models.Transformer("distilroberta-base") |
|
# Load and activate adapter |
|
word_embedding_model.auto_model.load_adapter("yoh/distilroberta-base-sept-adapter", source="hf", set_active=True) |
|
# Create sentence transformer |
|
pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension(), pooling_mode='mean') |
|
model = SentenceTransformer(modules=[word_embedding_model, pooling_model]) |
|
``` |
|
|
|
## Architecture & Training |
|
|
|
See this [paper](https://arxiv.org/abs/2311.00408) |
|
|
|
## Evaluation results |
|
|
|
See this [paper](https://arxiv.org/abs/2311.00408) |
|
|
|
## Citation |
|
```bibtex |
|
@article{huang2023adasent, |
|
title={AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification}, |
|
author={Yongxin Huang and Kexin Wang and Sourav Dutta and Raj Nath Patel and Goran Glavaš and Iryna Gurevych}, |
|
journal = {ArXiv preprint}, |
|
url = {https://arxiv.org/abs/2311.00408}, |
|
volume = {abs/2311.00408}, |
|
year={2023}, |
|
} |
|
``` |
