apache/spark — pull request #360 by AndreSchumacher (2014-04-08 15:20:24+00:00)
Title: SPARK-1293 [SQL] WIP Parquet support for nested types

It should be possible to import and export data stored in Parquet's columnar format that contains nested types. For example:
```java
message AddressBook {
required string owner;
optional group ownerPhoneNumbers {
repeated string values;
}
repeated group contacts {
required string name;
optional string phoneNumber;
}
}
```
The example could model a type (AddressBook) that contains records made of strings (owner), lists (ownerPhoneNumbers) and a table of contacts (e.g., a list of pairs of a map). The list of tasks are as follows:

Implement support for converting nested Parquet types to Spark/Catalyst types:

- [x] Structs
- [x] Lists
- [ ] Maps

Implement import (via ``parquetFile``) of nested Parquet types (first version in this PR)

- [x] Initial version (without maps)

Implement export (via ``saveAsParquetFile``)

- [ ] Initial version (missing)

Test support for AvroParquet, etc.

Example:
```scala
val data = TestSQLContext
.parquetFile("input.dir")
.toSchemaRDD
data.registerAsTable("data")
sql("SELECT owner, contacts[1].name FROM data").collect()
```