[SPARK-2483][SQL] Fix parsing of repeated, nested data access.

Author: Michael Armbrust <michael@databricks.com>

Closes #1411 from marmbrus/nestedRepeated and squashes the following commits:

044fa09 [Michael Armbrust] Fix parsing of repeated, nested data access.

(cherry picked from commit 0f98ef1a2c9ecf328f6c5918808fa5ca486e8afd)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SQL] Whitelist more Hive tests.

Author: Michael Armbrust <michael@databricks.com>

Closes #1396 from marmbrus/moreTests and squashes the following commits:

6660b60 [Michael Armbrust] Blacklist a test that requires DFS command.
8b6001c [Michael Armbrust] Add golden files.
ccd8f97 [Michael Armbrust] Whitelist more tests.

(cherry picked from commit bcd0c30c7eea4c50301cb732c733fdf4d4142060)
Signed-off-by: Michael Armbrust <michael@databricks.com>
SPARK-2407: Added internal implementation of SQL SUBSTR()

This replaces the Hive UDF for SUBSTR(ING) with an implementation in Catalyst
and adds tests to verify correct operation.

Author: William Benton <willb@redhat.com>

Closes #1359 from willb/internalSqlSubstring and squashes the following commits:

ccedc47 [William Benton] Fixed too-long line.
a30a037 [William Benton] replace view bounds with implicit parameters
ec35c80 [William Benton] Adds fixes from review:
4f3bfdb [William Benton] Added internal implementation of SQL SUBSTR()

(cherry picked from commit 61de65bc69f9a5fc396b76713193c6415436d452)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SQL] Attribute equality comparisons should be done by exprId.

Author: Michael Armbrust <michael@databricks.com>

Closes #1414 from marmbrus/exprIdResolution and squashes the following commits:

97b47bc [Michael Armbrust] Attribute equality comparisons should be done by exprId.

(cherry picked from commit 502f90782ad474e2630ed5be4d3c4be7dab09c34)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-2498] [SQL] Synchronize on a lock when using scala reflection inside data type objects.

JIRA ticket: https://issues.apache.org/jira/browse/SPARK-2498

Author: Zongheng Yang <zongheng.y@gmail.com>

Closes #1423 from concretevitamin/scala-ref-catalyst and squashes the following commits:

325a149 [Zongheng Yang] Synchronize on a lock when initializing data type objects in Catalyst.

(cherry picked from commit c2048a5165b270f5baf2003fdfef7bc6c5875715)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-2314][SQL] Override collect and take in JavaSchemaRDD, forwarding to SchemaRDD implementations.

Author: Aaron Staple <aaron.staple@gmail.com>

Closes #1421 from staple/SPARK-2314 and squashes the following commits:

73e04dc [Aaron Staple] [SPARK-2314] Override collect and take in JavaSchemaRDD, forwarding to SchemaRDD implementations.

(cherry picked from commit 90ca532a0fd95dc85cff8c5722d371e8368b2687)
Signed-off-by: Reynold Xin <rxin@apache.org>
[SPARK-2509][SQL] Add optimization for Substring.

`Substring` including `null` literal cases could be added to `NullPropagation`.

Author: Takuya UESHIN <ueshin@happy-camper.st>

Closes #1428 from ueshin/issues/SPARK-2509 and squashes the following commits:

d9eb85f [Takuya UESHIN] Add Substring cases to NullPropagation.

(cherry picked from commit 9b38b7c71352bb5e6d359515111ad9ca33299127)
Signed-off-by: Reynold Xin <rxin@apache.org>
[SPARK-2504][SQL] Fix nullability of Substring expression.

This is a follow-up of #1359 with nullability narrowing.

Author: Takuya UESHIN <ueshin@happy-camper.st>

Closes #1426 from ueshin/issues/SPARK-2504 and squashes the following commits:

5157832 [Takuya UESHIN] Remove unnecessary white spaces.
80958ac [Takuya UESHIN] Fix nullability of Substring expression.

(cherry picked from commit 632fb3d9a9ebb3d2218385403145d5b89c41c025)
Signed-off-by: Reynold Xin <rxin@apache.org>