BUG: TimeGrouper doesnt exclude the column specified by key

This solves latter exclude column issue described in #7227.