Server: Make code in build_fov_map() slightly less bizarre.