add note to docs about carefully handling pagination; add test cases for pagination and when rescoring penalizes top hits