fixed precision of calculations