Add a missing case for invlet reshuffling on force invlet assignment.

Fixes issue #2672 See commit for more details.

Worked this out with kevingranade, committing in his place since he's short on time.