Revert part of #3108 to avoid rendering garbage memory

Issue #3237

This should still maintain most of the fixes from  #3108