elasticsearch/elasticsearch — pull request #6124 by martijnvg (2014-05-12 10:44:11+00:00)
Title: Add top_hits aggregation

The `top_hits` aggregator keeps track of the most relevant document being aggregated. This aggregator should be used as a sub aggregator of a bucket based aggregator, so that the top documents per bucket are computed.

Via this aggregator [grouping / field collapsing](https://github.com/elasticsearch/elasticsearch/issues/256) can be achieved and is very versatile. Someone can group by a field (using a terms aggregator as parent) or by time (using a histogram aggregator as parent), in any case the parent bucket aggregator determines how to group. How correct the top hits will depend on the parent aggregator. For example when using the `terms` aggregator and the `top_hits` aggregator some document may not end up in the response, because the `shard_size` on the `terms` aggregator is less then the field's cardinality.

The `top_hits` aggregator should have the following options:
* `size` - The amount of hits to collect.
* `sort` - Defines how the top hits should be sorted.
* and any other fetch phase options. Like source filtering and highlighting.

The prototype that is attached right now to this PR integrates nicely with the fetch phase, which allows all fetch like features to be implemented easily. Also it executes as if the `search_type` is set to `query_and_fetch`, this way aggregations don't need to execute extra round trips.

Example usage of the current prototype:
```json
GET /stack/question/_search?search_type=count
{
"aggs": {
"terms": {
"terms": {
"field": "tags",
"size": 10
},
"aggs": {
"top_tag_hits": {
"top_hits": {
"_source": {
"include": [
"title"
]
},
"sort": [
{
"last_activity_date": {
"order": "desc"
}
}
],
"size" : 3
}
}
}
}
}
}
```

In this example the hits are sorted by the field `last_activity_date` and only the top 3 hits are returned. Also per hit only the `title` field is included.

Response:
```json
{
"took": 151,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 175275,
"max_score": 0,
"hits": []
},
"aggregations": {
"terms": {
"buckets": [
{
"key": "windows-7",
"doc_count": 25365,
"top_tag_hits": {
"hits": {
"total": 25365,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602679",
"_score": 1,
"_source": {
"title": "Windows port opening"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602570",
"_score": 1,
"_source": {
"title": "Counter Strike Screen Resolution"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602249",
"_score": 1,
"_source": {
"title": "Hardware error while burning DVD+"
}
}
]
}
}
},
{
"key": "linux",
"doc_count": 18342,
"top_tag_hits": {
"hits": {
"total": 18342,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602672",
"_score": 1,
"_source": {
"title": "Ubuntu RFID Screensaver lock-unlock"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "543625",
"_score": 1,
"_source": {
"title": "Linux Mint doesn't boot after creating a swap partition"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602434",
"_score": 1,
"_source": {
"title": "Is desktop pc support ssd and sata hard disk in one machine?"
}
}
]
}
}
},
{
"key": "windows",
"doc_count": 18119,
"top_tag_hits": {
"hits": {
"total": 18119,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602678",
"_score": 1,
"_source": {
"title": "If I change my computers date / time, what could be affected?"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "472446",
"_score": 1,
"_source": {
"title": "Remove the Browser ballot app from Windows 8"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "321988",
"_score": 1,
"_source": {
"title": "How do I determine if my Windows is 32-bit or 64-bit using a command?"
}
}
]
}
}
},
{
"key": "osx",
"doc_count": 10971,
"top_tag_hits": {
"hits": {
"total": 10971,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602680",
"_score": 1,
"_source": {
"title": "How to Install Google Chrome from the command line"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602482",
"_score": 1,
"_source": {
"title": "All Mac OS X apps crash as opened"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "517263",
"_score": 1,
"_source": {
"title": "Create a shortcut for application on Google Chrome for MacOSX"
}
}
]
}
}
},
{
"key": "ubuntu",
"doc_count": 8743,
"top_tag_hits": {
"hits": {
"total": 8743,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602665",
"_score": 1,
"_source": {
"title": "Add more partitions to Grub2 - Ubuntu"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "190759",
"_score": 1,
"_source": {
"title": "Ubuntu 10.04 Keyboard and Mouse Freezing Problem"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "597297",
"_score": 1,
"_source": {
"title": "Curly braces with LCtrl+LShift+LAlt+è - how?"
}
}
]
}
}
},
{
"key": "windows-xp",
"doc_count": 7517,
"top_tag_hits": {
"hits": {
"total": 7517,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602679",
"_score": 1,
"_source": {
"title": "Windows port opening"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "510161",
"_score": 1,
"_source": {
"title": "Windows 8 Hyper-V fails to boot Windows XP ISO"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "180565",
"_score": 1,
"_source": {
"title": "Logitech Optical Mouse Frozen In Middle of Windows XP Pro Screen"
}
}
]
}
}
},
{
"key": "networking",
"doc_count": 6739,
"top_tag_hits": {
"hits": {
"total": 6739,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602679",
"_score": 1,
"_source": {
"title": "Windows port opening"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602645",
"_score": 1,
"_source": {
"title": "are there any requirements for the sequence number on CP RST packets?"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602449",
"_score": 1,
"_source": {
"title": "Vmware Dev Server not allowing HTTP traffic"
}
}
]
}
}
},
{
"key": "mac",
"doc_count": 5590,
"top_tag_hits": {
"hits": {
"total": 5590,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "602482",
"_score": 1,
"_source": {
"title": "All Mac OS X apps crash as opened"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602553",
"_score": 1,
"_source": {
"title": "How can I load VLC instead of iTunes on my Mac when I press the player buttons on my Mac keyboard?"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602460",
"_score": 1,
"_source": {
"title": "startup mac mini using boot usb with MASTER BOOT RECORD scheme but failed"
}
}
]
}
}
},
{
"key": "wireless-networking",
"doc_count": 4409,
"top_tag_hits": {
"hits": {
"total": 4409,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "265142",
"_score": 1,
"_source": {
"title": "Connect to Wi-Fi access point with specific MAC address"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "602586",
"_score": 1,
"_source": {
"title": "How to adjust Tx Power for Macbook Air mid-2012 Wi-Fi card"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "435290",
"_score": 1,
"_source": {
"title": "Use wifi and ethernet simultaneously?"
}
}
]
}
}
},
{
"key": "windows-8",
"doc_count": 3601,
"top_tag_hits": {
"hits": {
"total": 3601,
"max_score": 1,
"hits": [
{
"_index": "stack",
"_type": "question",
"_id": "510161",
"_score": 1,
"_source": {
"title": "Windows 8 Hyper-V fails to boot Windows XP ISO"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "591388",
"_score": 1,
"_source": {
"title": "Android USB Driver on Windows 8"
}
},
{
"_index": "stack",
"_type": "question",
"_id": "553339",
"_score": 1,
"_source": {
"title": "Can I have 2 PDF documents open in Windows 8?"
}
}
]
}
}
}
]
}
}
}
```