Gossips are frogs, they drink and talk. If you don't say it you will not have to unsay it. It takes two to have an argument. Keep your mouth shut and your eyes open. Many a true word spoken in jest.
Silence is an excellent remedy against slander.
Silence is golden. Silence is less injurious than a bad reply. Speak clearly, if you speak at all. Stop beating around the bush. Take your wife's first advice.