Don't count him as an enemy, but admonish him as a brother.
Paul makes it clear the true believers are not to treat the erring person unkindly. There are to be no bitter words against him lest such would provoke him instead of making him ashamed. This action would defeat the very purpose of marking and avoiding him. Paul has a warm heart and a love for these brethren who would not work; but he cannot condone their laziness and is forced to do something about it.