I think we need a better fix for this: it's bad that we pass the parent aggregator in but it's not fully constructed yet. Additionally, I'm concerned that the equals method of `ClosestNestedParentFilter` is not symetric, this could lead to duplicates in the filter cache.