An extra mortgage payment is already costly enough, but there are other factors to consider, like: This includes other payments on your primary home. If your second home won’t be your primary residence, you’ll need to add costs for a home manager and other maintenance you won’t be able to do yourself.