The way to scale is to not stick it in pg_info_t.  I don't think it's wroth the effort, though; i'd rather enforce a limit (at the mon, maybe) that caps the number of hitsets the user is allowed to configure for a given pg.

Note that the pg_history_t can similarly grow large, so this isn't so unusual.  At least in this case though we can prevent the user from shooting themselves in the foot (whereas with history it's a function of how much peering has been thrashing).