apache/spark — pull request #2655 by scwf (2014-10-04 15:27:35+00:00)
Title: [SPARK-3793][SQL]use hiveconf when parse hive ql

This PR is to make hive ql parser more general and compatible with both hive-0.12 and hive-0.13. In hive-0.13 we may need hiveconf(or hivecontext) when parsing a sql(quoted sql). For example, when runing sql as follow without hiveconf will get NPE exception:

createQueryTest("quoted alias.attr",
"SELECT `a`.`key` FROM src a ORDER BY key LIMIT 1")
[info] - quoted alias.attr *** FAILED ***
[info] org.apache.spark.sql.hive.HiveQl$ParseException: Failed to parse: SELECT `a`.`key` FROM src a ORDER BY key LIMIT 1
[info] at org.apache.spark.sql.hive.HiveQl$.parseSql(HiveQl.scala:221)
[info] at org.apache.spark.sql.hive.test.TestHiveContext$HiveQLQueryExecution.logical$lzycompute(TestHive.scala:143)
[info] at org.apache.spark.sql.hive.test.TestHiveContext$HiveQLQueryExecution.logical(TestHive.scala:143)
[info] at org.apache.spark.sql.hive.test.TestHiveContext$QueryExecution.analyzed$lzycompute(TestHive.scala:153)
[info] at org.apache.spark.sql.hive.test.TestHiveContext$QueryExecution.analyzed(TestHive.scala:152)
[info] at org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan$lzycompute(SQLContext.scala:403)
[info] at org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan(SQLContext.scala:403)
[info] at org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan$lzycompute(SQLContext.scala:407)
[info] at org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan(SQLContext.scala:405)
[info] at org.apache.spark.sql.SQLContext$QueryExecution.executedPlan$lzycompute(SQLContext.scala:411)
[info] ...
[info] Cause: java.lang.NullPointerException:
[info] at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:1295)
[info] at org.apache.hadoop.hive.ql.parse.HiveLexer.allowQuotedId(HiveLexer.java:342)
[info] at org.apache.hadoop.hive.ql.parse.HiveLexer$DFA21.specialStateTransition(HiveLexer.java:10945)
[info] at org.antlr.runtime.DFA.predict(DFA.java:80)
[info] at org.apache.hadoop.hive.ql.parse.HiveLexer.mIdentifier(HiveLexer.java:7925)
[info] at org.apache.hadoop.hive.ql.parse.HiveLexer.mTokens(HiveLexer.java:10818)
[info] at org.antlr.runtime.Lexer.nextToken(Lexer.java:89)
[info] at org.antlr.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:133)
[info] at org.antlr.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:127)
[info] at org.antlr.runtime.CommonTokenStream.consume(CommonTokenStream.java:70)