You don't have fun together One of the big signs he's not the one is if you don't enjoy your time with him. Going out with your man should be one of the highlights of your week, not something you dread. If you spend more time arguing than enjoying each other's company, it's a sign he's just not right for you.