Repaying money originally documented as a gift is considered mortgage fraud, which involves lying or omitting information on a mortgage application to get a loan. If the lender or IRS finds out, it can have legal ramifications.