ReplicatedPG: create max hitset size

Otherwise, hit_set_create could create an unbounded size hitset
object.

Fixes: #9339
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2437 from athanatos/wip-9339

ReplicatedPG: create max hitset size

Reviewed-by: Sage Weil <sage@redhat.com>