twitter/scalding — pull request #1087 by amatsukawa (2014-10-28 01:02:49+00:00)
Title: Refactor JDBCSource to add compile-time info about type of DB

This builds off @jcoveney's PR #1010. I figured I'd leave his branch alone, since this takes a slightly different approach.

Since we've pushed the type of DB for a source to compile time, opted to turn drivers into traits. Put all driver-specific logic there. (not only generating the correct SQL for table creation, but also things like getting the correct JDBCSchema). `JDBCSourceCompileTest` is an example source that exercises this code. IMO this is less boilerplate than having the driver as a generic type.

I've also pulled out components of JDBCSource to its own file, since the one file was getting pretty gigantic.

[WIP] still need to do Vertica. Just thought I would get it out there for some comments.

Also had some offline discussions with @ianoc. This change requires that we change existing all JDBCSources, which is a lot of friction. Another possibility is to push SQL generation to run-time, after picking up the correct driver (currently the SQL to create the table is basically generated at compile time). This has the trade-off that

(positive) is a smaller change in API and allows us to swap databases
(negative) if a DB does not support a specific column or other such issues (mentioned by @jcoveney as an advantage to this approach in #1010) only manifest at run time.