DeiT models face computational cost challenges due to their reliance on multi-head self-attention modules and other complex components. Recent research has focused on improving DeiT's efficiency and performance by introducing novel techniques, such as Token Pruning & Squeezing (TPS) modules for compressing vision transformers more efficiently. These techniques aim to reduce the computational cost while maintaining or improving the accuracy of DeiT models.