GitHub repository: rugvedmhatre/Efficient-ViT
Primary language: Jupyter Notebook
Created: 2024-11-24T17:36:38Z
Description: Optimizing Vision Transformers (ViTs) for resource-constrained environments by reducing model size and inference latency while maintaining performance. Implements flash attention, attention head pruning, teacher-student distillation, and quantization, inspired by the Once-for-All approach for CNNs.