Issue 1:
Hi,

I'm face a interesting problem while executing alot of a simple query via head plugin in a little time (about ~ 20 clicks successive)

{
"size" : 90000000,
"query" : {
"term" : { "username" : "kimchy" }
},
"fields" : []
}

The result is the following (while Elasticsearch take time to anwser)

{
error: ReduceSearchPhaseException[Failed to execute phase [query], [reduce] ]; nested: OutOfMemoryError[Java heap space];
status: 503
}

But I have only 11 docs in my index.

Log debug trace

Failed to execute [org.elasticsearch.action.search.SearchRequest@3a44ebcb] while moving to second phase
java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:64)
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:37)
at org.elasticsearch.search.controller.ScoreDocQueue.<init>(ScoreDocQueue.java:31)
at org.elasticsearch.search.controller.SearchPhaseController.sortDocs(SearchPhaseController.java:257)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.moveToSecondPhase(TransportSearchQueryThenFetchAction.java:85)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.innerMoveToSecondPhase(TransportSearchTypeAction.java:417)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:241)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onResult(TransportSearchTypeAction.java:219)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onResult(TransportSearchTypeAction.java:216)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:203)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

a priori, in Apache Lucene PriorityQueue.class, this following line causing problem

heap = (T[]) new Object[heapSize];

Questions :

Could you look up for fixing this issue? heapSize should be equal

heapSize = Math.min(nDocsTotalInIndex, size); like it's did in IndexSearch class of Lucene (line 445)

Do you know why PriorityQueue pre-allocate a full array of length (will causing HeapSize or GC problem if we do a lot of query with high 'size' value)? Why don't simply use List < Object > ? Please clarify me, just for curiosity :)

Thanks

Setup:

Elasticsearch 1.0.1 release

java version "1.7.0_25"

Java(TM) SE Runtime Environment (build 1.7.0_25-b15)

Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

PS: maybe the same problem with this one 4177

Issue 2:
Hi,

I am executing a simple query (using the Chrome Postman App) on an index with around 80000 records, with a 'from' value which is way higher than the number of available pages.

{

"from" : 200000000,

"size" : 1,

"query" : {

"query_string" : {

"query" : "test"

}

}

}

This results in the following response:

{

"error" : "ReduceSearchPhaseException[Failed to execute phase [query], [reduce] ]; nested: OutOfMemoryError[Java heap space]; ",

"status" : 503

}

Interestingly, when I execute the same query from a Java client using the Java API, the same error occurs but the value can be much lower, around 2000000. Elasticsearch and Client running in separate JVMs,

Our web application fronting ElasticSearch restricts the page size but not the page number as I assumed this would always return 0 results if the client request exceeds the available number of pages. However, if the client provides a very high number, this OOM occurs.

Setup:

$ java -version

java version "1.7.0_17"

Java(TM) SE Runtime Environment (build 1.7.0_17-b02)

Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Windows 7

Elasticsearch 0.90.5 (standard setup, no settings changed after install)

ES LOG after query direct from postman:

[2013-11-15 14:05:04,323][INFO ][node ] [Guthrie, Jebediah] version[0.90.5], pid[14304], build[c8714e8/2013-09-17T12:50:20Z]

[2013-11-15 14:05:04,324][INFO ][node ] [Guthrie, Jebediah] initializing ...

[2013-11-15 14:05:04,330][INFO ][plugins ] [Guthrie, Jebediah] loaded [], sites []

[2013-11-15 14:05:07,601][INFO ][node ] [Guthrie, Jebediah] initialized

[2013-11-15 14:05:07,602][INFO ][node ] [Guthrie, Jebediah] starting ...

[2013-11-15 14:05:07,790][INFO ][transport ] [Guthrie, Jebediah] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.201.87.208:9300]}

[2013-11-15 14:05:11,006][INFO ][cluster.service ] [Guthrie, Jebediah] new_master [Guthrie, Jebediah][rwzTTiUKQXG7mX0YuYc3SQ][inet[/10.201.87.208:9300]], reason: zen-disco-join (elected_as_master)

[2013-11-15 14:05:11,079][INFO ][discovery ] [Guthrie, Jebediah] elasticsearch/rwzTTiUKQXG7mX0YuYc3SQ

[2013-11-15 14:05:11,228][INFO ][http ] [Guthrie, Jebediah] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.201.87.208:9200]}

[2013-11-15 14:05:11,229][INFO ][node ] [Guthrie, Jebediah] started

[2013-11-15 14:05:11,942][INFO ][gateway ] [Guthrie, Jebediah] recovered [2] indices into cluster_state

[2013-11-15 14:05:44,744][DEBUG][action.search.type ] [Guthrie, Jebediah] [gtr][0]: Failed to execute [org.elasticsearch.action.search.SearchRequest@2bc44fb7] while moving to second phase

java.lang.OutOfMemoryError: Java heap space

at org.apache.lucene.util.PriorityQueue.(PriorityQueue.java:64)

at org.apache.lucene.util.PriorityQueue.(PriorityQueue.java:37)

at org.elasticsearch.search.controller.ScoreDocQueue.(ScoreDocQueue.java:31)

at org.elasticsearch.search.controller.SearchPhaseController.sortDocs(SearchPhaseController.java:248)

at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.moveToSecondPhase(TransportSearchQueryThenFetchAction.java:85)

at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.innerMoveToSecondPhase(TransportSearchTypeAction.java:409)

at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:241)

at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onResult(TransportSearchTypeAction.java:219)

at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onResult(TransportSearchTypeAction.java:216)

at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:203)

at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:722)

[2013-11-15 14:05:46,165][INFO ][cluster.service ] [Guthrie, Jebediah] added {[Man-Spider][4F01Q8RwQHOhDW_LB2u1YQ][inet[/10.201.87.208:9301]]{data=false, local=false, master=false},}, reason: zen-disco-receive(join from node[[Man-Spider][4F01Q8RwQHOhDW_LB2u1YQ][inet[/10.201.87.208:9301]]{data=false, local=false, master=false}])

CLIENT Log using Java API:

org.elasticsearch.action.search.ReduceSearchPhaseException: Failed to execute phase [query], [reduce]

at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetchPhase(TransportSearchDfsQueryThenFetchAction.java:180) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$3.onResult(TransportSearchDfsQueryThenFetchAction.java:154) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$3.onResult(TransportSearchDfsQueryThenFetchAction.java:148) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.search.action.SearchServiceTransportAction$5.handleResponse(SearchServiceTransportAction.java:251) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.search.action.SearchServiceTransportAction$5.handleResponse(SearchServiceTransportAction.java:242) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:153) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:124) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[elasticsearch-0.90.5.jar:na]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_17]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_17]

at java.lang.Thread.run(Thread.java:722) ~[na:1.7.0_17]

java.lang.OutOfMemoryError: Java heap space

at org.apache.lucene.util.PriorityQueue.(PriorityQueue.java:64) ~[lucene-core-4.4.0.jar:4.4.0 1504776 - sarowe - 2013-07-19 02:53:42]

at org.apache.lucene.util.PriorityQueue.(PriorityQueue.java:37) ~[lucene-core-4.4.0.jar:4.4.0 1504776 - sarowe - 2013-07-19 02:53:42]

at org.elasticsearch.search.controller.ScoreDocQueue.(ScoreDocQueue.java:31) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.search.controller.SearchPhaseController.sortDocs(SearchPhaseController.java:248) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.innerExecuteFetchPhase(TransportSearchDfsQueryThenFetchAction.java:185) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetchPhase(TransportSearchDfsQueryThenFetchAction.java:178) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$3.onResult(TransportSearchDfsQueryThenFetchAction.java:154) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$3.onResult(TransportSearchDfsQueryThenFetchAction.java:148) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.search.action.SearchServiceTransportAction$5.handleResponse(SearchServiceTransportAction.java:251) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.search.action.SearchServiceTransportAction$5.handleResponse(SearchServiceTransportAction.java:242) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:153) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:124) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[elasticsearch-0.90.5.jar:na]

at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[elasticsearch-0.90.5.jar:na]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_17]