Fix setting pivot not properly

When all values were smaller than the pivot, it would fail, as it would
get the place to swap to from the first open card, which wasn’t there.
This provides a fix for that issue.
Also, when the last element was smaller than the pivot, it would forget
to switch that one too.