facebook/presto — pull request #950 by zhenxiao (2014-01-09 06:53:20+00:00)
Title: Fix NullPointerException when reading from S3

When presto reading from S3, get the following NullPointerException:

presto:benchmark> select dateint, hour, hostname, count(*) as cnt from streaming_client_log where dateint=20130701 GROUP BY dateint, hour, hostname order by cnt;

Query 20140109_060239_00014_ddzak, FAILED, 8 nodes
http://ec2-54-205-226-69.compute-1.amazonaws.com:8080/v1/query/20140109_060239_00014_ddzak?pretty
Splits: 10 total, 8 done (80.00%)
CPU Time: 0.0s total, 0 rows/s, 0B/s, 50% active
Per Node: 0.0 parallelism, 0 rows/s, 0B/s
Parallelism: 0.0
0:01 [0 rows, 0B] [0 rows/s, 0B/s]

Query 20140109_060239_00014_ddzak failed: null
java.lang.NullPointerException
at org.apache.hadoop.fs.BufferedFSInputStream.getPos(BufferedFSInputStream.java:50)
at org.apache.hadoop.fs.FSDataInputStream.getPos(FSDataInputStream.java:41)
at org.apache.hadoop.io.SequenceFile$Reader.getPosition(SequenceFile.java:2257)
at org.apache.hadoop.mapred.SequenceFileRecordReader.getProgress(SequenceFileRecordReader.java:114)
at com.facebook.presto.hive.GenericHiveRecordCursor.getCompletedBytes(GenericHiveRecordCursor.java:194)
at com.facebook.presto.operator.RecordProjectOperator.getOutput(RecordProjectOperator.java:157)
at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:201)
at com.facebook.presto.operator.Driver.process(Driver.java:214)
at com.facebook.presto.operator.Driver.processFor(Driver.java:243)
at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:636)
at com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:436)
at com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:570)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

This root reason is, in RecordProjectOperator.getOutput(), advanceNextPosition() will close the cursor when reaching the end of file, the following cursor.getCompletedBytes() will get NullPointerException, since cursor is already closed.

This patch fixes the bug.