Issue 1:
Trying to run a test that extends ESIntegTestCase with Elasticsearch 2.0.0, getting this:

java.lang.RuntimeException: found jar hell in test classpath
at org.elasticsearch.bootstrap.BootstrapForTesting.<clinit>(BootstrapForTesting.java:63)
at org.elasticsearch.test.ESTestCase.<clinit>(ESTestCase.java:106)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:573)
Caused by: java.lang.IllegalStateException: jar hell!
class: jdk.packager.services.UserJvmOptionsService
jar1: /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/lib/ant-javafx.jar
jar2: /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/lib/packager.jar
at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:267)
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:185)
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:86)
at org.elasticsearch.bootstrap.BootstrapForTesting.<clinit>(BootstrapForTesting.java:61)
... 4 more

Not sure I can do much about that.

Issue 2:
ES' hit highlighting abilities are really sophisticated, but I'd like to suggest the API be made a bit more flexible.

One use case we have is the user performs their query against the _all field, and our UI needs to highlight matching terms in whatever specific fields the user happens to be viewing.

The fact that the {"highlight"} API lets you wildcard field names is pretty great, until you simply specify "*". What happens is that ES will complain that certain fields aren't necessarily compatible with the highlighting options specified. For example, you might see:

[2014-04-09 16:48:07,009][DEBUG][action.search.type ] [Miles Warren] [6838] Failed to execute fetch phase
org.elasticsearch.ElasticsearchIllegalArgumentException: the field [edit_date] should be indexed with term vector with position offsets to be used with fast vector highlighter
at org.elasticsearch.search.highlight.FastVectorHighlighter.highlight(FastVectorHighlighter.java:68)
at org.elasticsearch.search.highlight.HighlightPhase.hitExecute(HighlightPhase.java:126)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:211)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:444)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:406)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:150)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$2.run(TransportSearchQueryThenFetchAction.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

("edit_date" is defined as a date field and of course has no "index_options" on it).

And if you hack a bit at the ES sources to silently ignore these errors you'll see it also complains about ES' core fields such as _id, _timestamp, _source, etc.

What I'd like to propose is an option in the { "highlight" } API to be lenient about mismatches between field mapping definitions and highlight parameters. So if the highlight parameters aren't compatible with the mapping, ES will just move on to the next field, rather than bomb the entire request.

Call it a best-efforts highlighting for "all" fields.

Maybe it could fall back to decreasingly less sophisticated highlighters, until it just can't do anything. So if the request indicates the fvh be used, and that's not compatible with a particular field, it tries the postings highlighter, then the plain highlighter, then nothing.

Additionally, I'd like to see a highlight parameter to exclude fields. Another use case we have is, "I want to highlight all matched fields (whatever they are), except this array of 'large text' fields that I'm not currently showing the user" -- this would really be a bit of a performance optimization, from the client. Our data model includes a couple of large text fields (10k+) per document that wouldn't need to be highlighted except at very specific times -- the other fields are small and generally need to be highlighted in one call to ES.

Thoughts?

If y'all think this is a good idea, I'm happy to fork, do the work, make a pull request, and work with the committer(s) until it's accepted. I basically need to do this anyways, so I'd rather have buy-in instead of maintaining my own patched version of ES.

I thought about doing this as a plugin, but it looked like I'd have to duplicate quite a bit of core ES classes, which doesn't seem maintainable in the long run.