Regularization aims to reduce the variance of the estimator by virtue of simplifying it. This will increase the bias in a manner that will cause the expected error to rise. This is usually done when the problem is not posed well, like situations where the number of parameters is more than the number of samples.