Not at all. Fannie and Freddie both welcome seasoned (repeat) buyers. But not all HFAs do, so check your local guidelines. These types of programs often define a first-time home buyer as someone who has not owned a home in the last 3 years.