cloudera/Impala — pull request #67 by mangonc (2014-10-08 09:52:13+00:00)
Title: Why impala takes so much time to fetch rows?

I checked that the major time of the query latency is related with fetch rows. We can see that query is FINISHED in x secounds but only ends much time later depending on the number or rows fetched.

After several testes, I checked that:
- The query latency when executed on impala-shell is not afected by fetch rows.
- But the same query executed via JDBC on same host that impala-shell, takes very much time to end.
- The JDBC fetchSize can´t be set to a value greater than 1024 rows. Change the batch_size on impalad server also no takes effect.
- One diference of the impala-shell relatively to JDBC is the port used 21000 and 21050 respectively.

But even on jdbc if I concat all columns in only one column the latency of the query is similar to the time of the same query executed via impala-shell. I think that exist some weight in processing of the columns that causes the increase of the latency. It is affected depending on the number of columns and rows.

Can someone explain why this happens?