The man of God should not only hold the truth, he should also refute error. Certainly this means that discerning Christians should not be a part of denominational fellowships or other religious organizations which condone or tolerate unbelieving apostates in them. While some would allow that one ought to separate from unbelievers, they nevertheless claim that one believer should never break fellowship with another believer. But the Bible is clear that there are conditions under which a believer is not to fellowship with another believer. Immoral believers are to be disciplined.
Church discipline includes separation from other believers.
First Corinthians 5 is devoted to this matter. Therefore put away from among yourselves that wicked person. Sometimes a believer must break fellowship with another believer. Disorderly believers are to be avoided. The disorderliness is in the area of practical application of truth which had been taught.