EVR renderers: Fix high CPU and memory usage when switching the user account.

This commit fixes #3176.