Let's  He is married, they have no business reasons to get together, the whole office is chirping about their lengthy closed-door meetings, they whisper, leave to have lunch together, and you think there is no more than a flirtation.
He is married and people think he is acting inappropriately.