Properly update markers list when markers are removed and added