[RFC] Fix some more -Wconversion warnings.

This is on top of #1488, but I'll rebase as soon as that one is merged.
Looks like `l` can be moved here.