Yes. The controllers affect your Beat Saber gameplay. Here are some recommendations to get the best controllers:
Quest 2 and Rift S uses Oculus Touch controllers. They are compact, lightweight controllers. For so, they are comfortable to behold for a long time.
The Index controllers sometimes hinder wrist movement during your Beat Saber play. Therefore, you may need to take off the straps.
Vive wands are thicker than Oculus Touch, but they feel organic and aligned. The wand works well with sword VR games, especially in Beat Saber.