fixed recursive crawler ignores None values