Fix the render_labels logic

Fixes #409. I have no idea why I made that logic!