elasticsearch/elasticsearch — Issue #1593: Highlight snippets have whitespace appended when using term_vector:with_positions_offsets
Author: brianmario
Created: 2012-01-06 23:22:03+00:00
State: open

When configuring and searching the index with the following:

```
curl -XPUT http://localhost:9200/gists -d '{
"settings" : {
"analysis" : {
"analyzer" : {
"standard_nostop" : {
"tokenizer" : "standard",
"filter" : "standard,lowercase"
},
"standard_nostop_nolower" : {
"tokenizer" : "standard",
"filter" : "standard"
}
}
}
},
"mappings" : {
"gist" : {
"properties" : {
"body" : {
"type" : "multi_field",
"fields" : {
"body" : {
"type" : "string",
"analyzer" : "standard_nostop",
"term_vector" : "with_positions_offsets"
},
"nocase" : {
"type" : "string",
"analyzer" : "standard_nostop_nolower",
"term_vector" : "with_positions_offsets"
}
}
}
}
}
}
}'

curl -XPUT http://localhost:9200/gists/gist/1 -d '{
"body" : "Yajl::Parser.parse(str, :symbolize_keys => true)"
}'

curl http://localhost:9200/gists/gist/_search?pretty=1 -d '{
"query" : {
"dis_max" : {
"queries" : [
{"text_phrase" : {"body" : {"query" : "Yajl::Parser.parse", "boost" : 20}}},
{"text_phrase" : {"body.nocase" : {"query" : "Yajl::Parser.parse", "boost" : 30}}},
{"text" : {"body" : {"query" : "Yajl::Parser.parse", "boost" : 1}}},
{"text" : {"body.nocase" : {"query" : "Yajl::Parser.parse", "boost" : 3}}}
]
}
},
"highlight" : {
"fields" : {
"body" : {},
"body.nocase" : {}
}
}
}'
```

I get this result:

```
{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.26849622,
"hits" : [ {
"_index" : "gists",
"_type" : "gist",
"_id" : "1",
"_score" : 0.26849622, "_source" : {
"body" : "Yajl::Parser.parse(str, :symbolize_keys => true)"
},
"highlight" : {
"body" : [ "Yajl::Parser.parse(str, :symbolize_keys => true) " ],
"body.nocase" : [ "Yajl::Parser.parse(str, :symbolize_keys => true) " ]
}
} ]
}
}
```

Notice "body" and "body.nocase" both have an extra space appended to the end. Creating the index without {"term_vector" : "with_positions_offsets"} set fixes it.