Small fix for #4540 in empty maps

Fix #4540 by also checking if the map is empty