Cascading/cascading-jdbc — pull request #23 by locked-fg (2014-09-03 13:34:57+00:00)
Title: Enable MySQL Streaming

In Issue #21 the problem was that MySQL doesn't stream results.
The key issue here is that statement.setFetchSize(Integer.MIN_VALUE); needs to be performed on the statement in DBInputFormat's inner class DbRecordReader. And in the DBRecordReader's constructor there is the required change.

Before, the MySql client first read the complete result into ram and then handed it over to cascading. What you want is that results are streamed through the connection directly into cascading.

What I did:
- extract the "create statement" part into a method "createStatement()" so that it can be overriden by subclasses of DBRecordReader
- Created MySqlDBInputFormat which subclasses DBRecordReader and overloads the createStatement() and returns the according class (in getRecordReaderInternal)
- change MySqlFactory so that it returns the new MySqlDBInputFormat class