The input checking looks quite complicated. Can't this be simplified? Also, I would put it in a separate method / function.

By the way, if all solvers support lists of penalties, to simplify things, we can just pass a list of length 1 to ridge_regression, in the single target case.