Issue 1:
It looks like the actual document data indexed for the field "nums" is a list of 4 floats in descending order, but when the aggregation is run to collect the data, only 3 floats are returned and they are in ascending order.

#! /bin/sh

# Delete the index
curl -XDELETE "http://localhost:9200/test"

# Insert a document with one field -- a list of floats
curl -XPUT "http://localhost:9200/test/doc/1?refresh=true" -d '{
"nums": [5.0, 3.5, 3.5, 1.0]
}'

# Expected:
# aggregations.foobar.value == [5.0, 3.5, 3.5, 1.0]

# Actual:
# aggregations.foobar.value == [1.0, 3.5, 5.0]
curl -XGET "http://localhost:9200/test/_search" -d '{
"query": {
"match_all": {}
},
"filter": {},
"aggregations": {
"foobar": {
"scripted_metric": {
"init_script": "_agg[\"numList\"] = []",
"map_script": "if (doc != null) { _agg.numList.add(doc[\"nums\"].getValues()) }",
"combine_script": "return _agg.numList",
"reduce_script": "return _aggs.flatten()"
}
}
}
}'

Issue 2:
We are updating some data in our els store. The data has a creation timestamp so we fetch buckets of 1 hour length from the database. Therefore we need to fetch all items from this date bucket. Since we don´t know how many items could be in the bucket, we se the size of search to Integer.MAX_VALUE.

We´re seeing a dramatic drop in performance when setting the hit size that high. When we set the size closer to the actual size, the performance of the request is again expectably fast.

Scroll to my comment for a solution using the scroll API

we tried to reproduce the problem as well using the REST API directly:

With Integer.MAX_VALUE (POST {{elastic_search_host}}/resolve_log_v1/entry/_search?pretty)

{
"size" : 2147483647,
"query" : {
"bool" : {
"must" : [
{ "range" : { "trigger.timestamp" : { "gte": "2015-07-26T16:00:00.000Z", "lte" : "2015-07-26T17:00:00.000Z" } } }
]
}
}
}

Result:

{
"took": 22856,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 21,
"max_score": 1,
"hits": [

same request with size set to 21

{
"took": 16,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 21,
"max_score": 1,
"hits": [
{
"_index": "resolve_log_v1",
"_type": "entry",
"_id": "4851a61c-3a17-494d-b72e-9bce172d45e6",

when setting the size to 0 we still get a quick response.

The total count of entry objects (:

curl -X POST -H "Content-Type: application/json" -d '{ "size" : 0 }' 'http://localhost:9222/resolve_log_v1/entry/_search?pretty':
{
"took": 26,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1022824,
"max_score": 0,
"hits": []
}
}

To optimize, we now do two queries, 1 count, one to geht the hits:

BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery().must(rangeQuery("trigger.timestamp").gte(fromJoda.toString()).lte(toJoda.toString()));
SearchResponse sizeResponse = client.prepareSearch(SearchProvider.INDEX)
.setTypes(TYPE)
.setSearchType(SearchType.COUNT)
.setQuery(queryBuilder)
.execute()
.actionGet();

SearchResponse response = client.prepareSearch(SearchProvider.INDEX)
.setTypes(TYPE)
.setQuery(queryBuilder)
.setSize((int) sizeResponse.getHits().getTotalHits())
.execute()
.actionGet();

this is not ideal, as the bucket theoretically could change.