ENH: Improving speed of B matrices computation