While most FHA borrowers can continue to make down payments of as little as 3.5 percent when they take out a loan , those with a credit score of less than 580 will have to make a down payment of at least 10 percent , possibly starting in the early summer .