Yeah, I meant the same column appears multiple times in different
clauses...so if Optiq is taking care of it then I suppose you don't need to
check for the duplicates or the constant exprs.   The only other comment I
had was this is actually column push-down, not project push-down since you
are not getting rid of the Project Rels from the plan.  In theory, after
the columns have been pushed into the scan, if the Project above the Scan
is producing the exact same set of columns as the scan, then there is no
reason to keep the Project.  Do we want to address this now or defer it for
later ?


On Tue, Apr 15, 2014 at 10:55 AM, jinfengni <notifications@github.com>wrote:

> In
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrel.java:
>
> > @@ -54,7 +57,24 @@ protected Object clone() throws CloneNotSupportedException {
> >    @Override
> >    public PhysicalOperator getPhysicalOperator(PhysicalPlanCreator creator) throws IOException {
> >      StoragePlugin plugin = this.drillTable.getPlugin();
> > -    GroupScan scan = plugin.getPhysicalScan(new JSONOptions(drillTable.getSelection()));
> > +
> > +    boolean containStar = false;
> > +    final List<String> fields = getRowType().getFieldNames();
> > +    List<SchemaPath> columns = Lists.newArrayList();
> > +    for (String field : fields) {
>
> By "duplicate columns", do you mean the same query appears multiple time
> in Select clause, or order by clause, group by, etc...? If that's the case,
> optiq will take care of the duplicate columns in ProjectRel, SortRel, etc,
> since optiq uses index based RexNode to represent the expression. The Scan
> RelNode's RowType will have only the identical columns. For example, for
> this query: select R_REGIONKEY, R_NAME, R_NAME, R_REGIONKEY, R_REGIONKEY +
> 100 FROM REGION... Scan will have RowType (R_REGIONKEY, R_NAME), while
> ProjectRel will have different expressions referring to the same index in
> the input's RowType. DrillProjectRel(R_REGIONKEY=[$0], R_NAME=[$1],
> R_NAME0=[$1], R_REGIONKEY0=[$0], EXPR$4=[+($0, 100)]) For constant
> expressions, ProjectRel will take care of the constant expression. ScanPrel
> will only have to deal with column fields.
> ... <#1456687fb3827ffb_>
> On Tue, Apr 15, 2014 at 9:07 AM, amansinha100 <notifications@github.com>wrote:
> In
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrel.java:
> > @@ -54,7 +57,24 @@ protected Object clone() throws
> CloneNotSupportedException { > @Override <https://github.com/Override> >
> public PhysicalOperator getPhysicalOperator(PhysicalPlanCreator creator)
> throws IOException { > StoragePlugin plugin = this.drillTable.getPlugin();
> > - GroupScan scan = plugin.getPhysicalScan(new
> JSONOptions(drillTable.getSelection())); > + > + boolean containStar =
> false; > + final List<String> fields = getRowType().getFieldNames(); > +
> List<SchemaPath> columns = Lists.newArrayList(); > + for (String field :
> fields) { At the time of adding the columns, I think we should check for
> duplicates since we don't want duplicate columns to be produced by the
> underlying scan. Also, I am wondering if we should check for constant
> expressions...or is that already taken care of before getting here. -- Reply
> to this email directly or view it on GitHub<
> https://github.com/mapr/incubator-drill/pull/1/files#r11642143> .
>
> --
> Reply to this email directly or view it on GitHub<https://github.com/mapr/incubator-drill/pull/1/files#r11647820>
> .
>