frankscholten/mahout — comment on commit 779fa4aa98f16025b550fe274692e8660aa5dcce
File: integration/src/main/java/org/apache/mahout/text/LuceneIndexToSequenceFiles.java line 110
By s1monw:

you pass Int.MAX to the searcher which will create an int[Integer.MAX_VALUE] + a PriorityQueue of the same size. What I would do here instead of using the IndexSearcher is executing the query "manually". This is going to be a longer discussion :) but in pseudo code its this:

1. call ```Weight weight = IndexSearcher#createNormalizedWeight(query)```

2. get a scorer for each reader and consume the docs:

```
for (int i = 0; i < subReaders.length; i++) { // search each subreader

final Scorer scorer = (filter == null) ?
weight.scorer(subReaders[i], false, true) :
FilteredQuery.getFilteredScorer(subReaders[i], getSimilarity(), weight, weight, filter);
if (scorer != null) {
int doc;
while ((doc = nextDoc()) != NO_MORE_DOCS) {
// do something with the doc
Document theDoc = subReaders[i].doc(doc);
}
}
}
```