Avoid allocating bg temporary frame buffer, unless recalc is done