The answer depends on what speeds and latencies we're talking about. Games tend to be sensitive to latency, where lower is better. A good low-latency DDR4 kit is still a viable gaming option. However, the real-world performance differences are small and non-existent in GPU-limited scenarios, such as 4K gaming, which is usually the case unless you're chasing very high frames per second.
If you have a good DDR4 kit, let's say a 2x16GB DDR4-3200 C14 kit or better, there's little reason to spend the money to upgrade to DDR5 for gaming.
The higher bandwidth of DDR5 is beneficial in other areas though, particularly in creative-type applications. DDR5 uses less power, 32GB is generally the baseline, and a DDR5 system is better for future-proofing. As time goes on, we can expect faster and lower latency kits.