modified way to handle null observations and refined few sections