Yes, you should keep your VPN on all the time, but there are a few times when you shouldn't.
It's the only way to be sure that your real location and IP address are hidden, that your online data is encrypted, and that your ISP and other third parties can't see your browsing history.