Yes. Even if you wipe out all the fortress guards outside, you still need that crappy armor. These are dedicated sentries. Not even death keeps them from doing their job.