---
license: apache-2.0
tags:
- image-classification
datasets:
- Face-Mask18K 
---

# Distilled Data-efficient Image Transformer for Face Mask Detection

Distilled data-efficient Image Transformer (DeiT) model pre-trained and fine-tuned on Self Currated Custom Face-Mask18K Dataset (18k images, 2 classes) at resolution 224x224. It was first introduced in the paper Training data-efficient image transformers & distillation through attention by Touvron et al.  

## Model description

This model is a distilled Vision Transformer (ViT). It uses a distillation token, besides the class token, to effectively learn from a teacher (CNN) during both pre-training and fine-tuning. The distillation token is learned through backpropagation, by interacting with the class ([CLS]) and patch tokens through the self-attention layers.

Images are presented to the model as a sequence of fixed-size patches (resolution 16x16), which are linearly embedded. 

## Training Metrics
    epoch                    =          2.0
    total_flos               = 2078245655GF
    train_loss               =       0.0438
    train_runtime            =   1:37:16.87
    train_samples_per_second =        9.887
    train_steps_per_second   =        0.309

---

## Evaluation Metrics
    epoch                   =        2.0
    eval_accuracy           =     0.9922
    eval_loss               =     0.0271
    eval_runtime            = 0:03:17.36
    eval_samples_per_second =      18.22
    eval_steps_per_second   =       2.28