Don't sort files by match in the switch buffer. Maintain the lru order