One potential limitation of the quasi-orthogonal fine-tuning approach is the increased complexity introduced by the additional parameters for flexible norms and angular adjustments. This complexity may lead to longer training times and increased computational costs. To address this limitation, future research could focus on optimizing the regularization techniques and parameter tuning strategies to reduce the computational overhead while maintaining the adaptability of the model. Additionally, the methods may face challenges in handling extremely large or diverse datasets where the fine-tuning process may require more sophisticated regularization techniques to prevent overfitting or loss of generalization.