Package: adapter-transformers
IMPORTANT NOTEThis is the legacyadapter-transformerslibrary, which has been replaced by the newAdapters library, found here:https://github.com/adapter-hub/adapters.Install the new library via pip:pip install adapters.This repository is kept for archival purposes, and will not be updated in the future.
Please use the new library for all active projects.The documentation of this library can be found athttps://docs-legacy.adapterhub.ml.
The documentation of the newAdapterslibrary can be found athttps://docs.adapterhub.ml.
For transitioning, please read:https://docs.adapterhub.ml/transitioning.html.adapter-transformersA friendly fork of HuggingFace'sTransformers, adding Adapters to PyTorch language modelsadapter-transformersis an extension ofHuggingFace's Transformerslibrary, integrating adapters into state-of-the-art language models by incorporatingAdapterHub, a central repository for pre-trained adapter modules.💡 Important: This library can be used as a drop-in replacement for HuggingFace Transformers and regularly synchronizes new upstream changes.
Thus, most files in this repository are direct copies from the HuggingFace Transformers source, modified only with changes required for the adapter implementations.Installationadapter-transformerscurrently supportsPython 3.8+andPyTorch 1.12.1+.
Afterinstalling PyTorch, you can installadapter-transformersfrom PyPI ...pip install -U adapter-transformers... or from source by cloning the repository:git clone https://github.com/adapter-hub/adapter-transformers.git
cd adapter-transformers
pip install .Getting StartedHuggingFace's great documentation on getting started withTransformerscan be foundhere.adapter-transformersis fully compatible withTransformers.To get started with adapters, refer to these locations:Colab notebook tutorials, a series notebooks providing an introduction to all the main concepts of (adapter-)transformers and AdapterHubhttps://docs-legacy.adapterhub.ml, our documentation on training and using adapters withadapter-transformershttps://adapterhub.mlto explore available pre-trained adapter modules and share your own adaptersExamples folderof this repository containing HuggingFace's example training scripts, many adapted for training adaptersImplemented MethodsCurrently, adapter-transformers integrates all architectures and methods listed below:MethodPaper(s)Quick LinksBottleneck adaptersHoulsby et al. (2019)Bapna and Firat (2019)Quickstart,NotebookAdapterFusionPfeiffer et al. (2021)Docs: Training,NotebookMAD-X,Invertible adaptersPfeiffer et al. (2020)NotebookAdapterDropRücklé et al. (2021)NotebookMAD-X 2.0,Embedding trainingPfeiffer et al. (2021)Docs: Embeddings,NotebookPrefix TuningLi and Liang (2021)DocsParallel adapters,Mix-and-Match adaptersHe et al. (2021)DocsCompacterMahabadi et al. (2021)DocsLoRAHu et al. (2021)Docs(IA)^3Liu et al. (2022)DocsUniPELTMao et al. (2022)DocsSupported ModelsWe currently support the PyTorch versions of all models listed on theModel Overviewpagein our documentation.CitationIf you use this library for your work, please consider citing our paperAdapterHub: A Framework for Adapting Transformers:@inproceedings{pfeiffer2020AdapterHub,
    title={AdapterHub: A Framework for Adapting Transformers},
    author={Pfeiffer, Jonas and
            R{\"u}ckl{\'e}, Andreas and
            Poth, Clifton and
            Kamath, Aishwarya and
            Vuli{\'c}, Ivan and
            Ruder, Sebastian and
            Cho, Kyunghyun and
            Gurevych, Iryna},
    booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
    pages={46--54},
    year={2020}
}