Generally speaking, the answer is “yes” because a VPN adds complexity (and security) to your internet connection.
However, if you are based in China and you want to browse websites that are hosted outside of China, you might see the opposite effect, i.e. a faster download (and upload) speed.