I managed to decrease the CPU usage from 80% to 50% by making the drawing thread handling use wait and notify instead of checking booleans during each frame.