James feels that he can't trust a person who would tell such a lie, and questions their place in the movement.
Sexual infidelity has a pretty long history, and so does lying about it.
Neither is it confined to the male sex, even if James hasn't heard of any such occurrences. Most people would be reluctant to rush in to such a private area as the relationship between lovers. By taking up the cudgels in defense of the sexually deceived, James succeeds in making himself look like a sort of White Knight of Monogamy, or perhaps a Big Brother of the Bedroom. I'm concerned that this sort of immoderate rhetoric can give an impression of cultism or fanaticism about nonviolence in general. If people feel that unrealistic and abnormal standards of sexual conduct apply in the movement, and that they risk being confronted about behaviour which would be considered extremely private elsewhere, then they won't participate and the movement will be the poorer for their absence.