marmbrus/spark — pull request #4 by gzm0 (2014-07-08 11:49:10+00:00)
Title: Remove intermediate map for records. Allow serialization

As discussed via mail. Note that you'll need to update your local publish of refined-records for this to work.

A couple of comments:

- The query parser didn't like the newlines in the join test (error below)
- We still box, since the records use a fully generic implementation. If this is not acceptable, we should have a look at specialization, rather than implementing `getXXX` methods IMHO.
- Due to the `getXXX` methods on `Row`, I had to implement the data access with a pattern match. This might be slower than a Hashtable lookup if there are many fields.

```
[error] spark/sql/core/src/test/scala/org/apache/spark/sql/TypedSqlSuite.scala:47: exception during macro expansion:
[error] java.lang.RuntimeException: [1.59] failure: ``UNION'' expected but ErrorToken(illegal character) found
[error]
[error] SELECT a.name FROM table0 a JOIN table1 b ON a.age = b.age\n
[error] ^
[error]	at scala.sys.package$.error(package.scala:27)
[error]	at org.apache.spark.sql.catalyst.SqlParser.apply(SqlParser.scala:60)
[error]	at org.apache.spark.sql.SQLMacros$.sqlImpl(TypedSql.scala:78)
[error] val results = sql"""SELECT a.name FROM $people a JOIN $people b ON a.age = b.age
[error] ^
```