Support for filtering by joined table outside of select

Filtering by a joined table that is not part of the select would cause a KeyError when using aggregate_rows