Hugging Face's logo

antoinelouis
/
camembert-L10

Feature Extraction
Transformers
PyTorch
Safetensors
French
camembert
text-embeddings-inference
Model card Files Community
1
camembert-L10 / README.md
antoinelouis's picture
antoinelouis
Update README.md
8535a3e
preview code
|
raw
history blame
No virus
1.75 kB
metadata
license: apache-2.0
language:
  - fr
library_name: transformers
inference: false

CamemBERT-L10

This model is a pruned version of the pre-trained CamemBERT checkpoint, obtained by dropping the top-layers from the original model.

Usage 

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained('antoinelouis/camembert-L10')
model = AutoModel.from_pretrained('antoinelouis/camembert-L10')

Comparison

Model #Params Size Pruning
CamemBERT 110.6M 445MB -
CamemBERT-L10 96.4M 386MB -13%
CamemBERT-L8 82.3M 329MB -26%
CamemBERT-L6 68.1M 272MB -38%
CamemBERT-L4 53.9M 216MB -51%
CamemBERT-L2 39.7M 159MB -64%

Citation 

@online{louis2023,
   author    = 'Antoine Louis',
   title     = 'CamemBERT-L10: A Pruned Version of CamemBERT',
   publisher = 'Hugging Face',
   month     = 'october',
   year      = '2023',
   url       = 'https://huggingface.co/antoinelouis/camembert-L10',
}