The biggest downsides of current consumer VR headsets include cost, weight, visual artifacts like "screen door effect," motion sickness for some users, and the hassle of setup. As the technology matures, many of these issues are diminishing. But there are still tradeoffs versus non-VR gaming. You have to decide if total immersion is worth the premium price and setup effort needed in your personal case.