Remove optimization that was required with whiny nils

Whiny nils is no longer a thing, so we no longer need this optimization
Merge pull request #15536 from sgrif/sg-remove-whiny-nil-thing

Remove optimization that was required with whiny nils