Your credit score, loan-to-value ratio, loan duration, and down-payment amount are some of the things that directly affect the rates you get from lenders.