Clean up logic to decide whether to recalculate beam_fly