[backport] Fix limit case for non existant neighbors