Yes, the gift giver can sue you for a gift that they gave you. But that doesn’t mean a jury will agree with him. Gifts are not something that you have to give back or repay legally. If someone sues you over a gift then he has to show a jury that they were not gifts but some sorts of loans.