BATCH-1613: MultiThreadedFlatFileItemReader and FlatFilePartitioner implementations.
Recently, we have faced a need to prepare a solution for speeding up Spring Batch file reading process. Since the standard FlatFileItemReader implementation is not intended for use in a multi-threaded environment, we have looked at a possibly existing solution and found the discussion on Spring Batch forum:
http://forum.springsource.org/showthread.php?93151-multi-threaded-FlatFileItemReader
and the https://jira.springsource.org/browse/BATCH-1613 Jira issue.
Basing on the code provided by Stephane Nicoll in the forum I have prepared a pull request for your consideration.
The MultiThreadedFlatFileItemReader and FlatFilePartitioner have been reworked in order to improve performance
Below you can see the performance comparison on a 8 core MacBook Pro with an SSD drive.
The test (which can be found in the pull request) only reads items from a temporary flat file.
Though applying multiple threads for reading does not bring much in terms of the reading performance, as can be seen below, it can be useful when processing or writing step is blocking on an intensive I/O. The test simulated 10ms delay by simply calling Thread.sleep(10).

The improvements to the initial implementation include:
- Underlying file system seek function utilized in order to locate the starting point for a reading thread, so that partition processing threads do not need to read forward to the starting point;
- The partitioning occurs on the same first fast reading sweep over the file (measured 2 to 5 times faster performance in comparison to BufferedReader and LineNumberReader implementation) at the line borders alowing further improvements supporting multi-line records;
- Partitioner distributes data between partitions more evenly to avoid having a partition twice as big as the others in an unlucky case
- Some unit tests added

Performance measured on a MacBook Pro i7 8 cores with SSD drive and on
a Intel Q6600 2,4GHz 4 cores CPU with ordinary HDD.
Input file 15232 lines (4Mb+)
Results are in milliseconds average of 4 attempts

Current implementation:
-----------------------
<pre>
Cores Threads 	No delay (log off); No delay (log DEBUG); Delay 10ms (log DEBUG);

	8 		1 		6003			7595		   171924
			2 		3531			5909			86172
			4 		3154 			5562			43291
			8 		3329 			5583			21979
		   16 			3624 			5670			11492

	4 		1 		8554 		   10734		       188855
			2 		5402 			8235			98238
			4 		5093 			8057			44679
			8 		5115			8398			21623
		   16	   		5318 			8389			11990
</pre>

Initial implementation (submitted in the forum):
------------------------------------------------
<pre>
Cores Threads 	No delay (log off); No delay (log DEBUG); Delay 10ms (log DEBUG);

	8 		1 		6063			9656		   172209
			2 		4420			9138			87852
			4 		4579			9833			45966
			8 		6193		   11680			26536
		   16 			9248						20492

</pre>