The down payment isn't the only piece of the home buying puzzle you'll need to figure out. You'll also need to be ready to pay closing costs: lender fees, taxes, appraisal fees, settlement charges, and more. These add up quickly. In 2021, average closing costs were $6,905, according to ClosingCorp.