Fix only consider number units if numbers have actually be found before