[backend] also fill remotemap for aggregates, otherwise partitioned aggregates do not work