D49069: lucene-solr: adds BBZKUploader class
D49073: lucene-solr: adds ZkServerManager class
lucene: Enhance xml/CoreParser support

 - Add PhraseQuery support
 - Term support from TermQuery
 - Split up CorePlusExtensionsParser
solr: Add XmlQParserPlugin support
lucene-solr: Add support for Term[s]Freq(Filter|Query)
[for discussion] lucene: remove repeated nl.getLength() calls in ...Builder classes

Summary: BooleanFilterBuilder, BooleanQueryBuilder, DisjunctionMaxQueryBuilder, FuzzyLikeThisQueryBuilder, TermBuilder, BoostedQueryBuilder are the builder classes. This is change is potential follow-on from D85313 findings.

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D85937
SOLR-6067: add buildAndRunCollectorChain method to refactor code in SolrIndexSearcher/Grouping

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D78182
Differential Revision: https://all.phab.dev.bloomberg.com/D80502
Differential Revision: https://all.phab.dev.bloomberg.com/D81470
solr: Pull in lucene intervals change

This contains the core changes needed for intervals, stripping off any maven build additions
originally at https://github.com/bloomberg/lucene-solr/commit/423ef0a20ef652eccf49148eb30e5a3f201df626
Further fixes to the upstream flaxsearch/lucene-solr-intervals have been folded in as well..
Lucene Intervals: Mark BadApples, Ignore Lucene3x codecs, fix link in JavaDoc
solr: DebugComponent debug.track output tweaks (SimpleOrderedMap instead of NamedList, shardAddress instead of shard, numeric values as such instead of string)

Summary:
For our use only, not for JIRA.

before:

{"responseHeader":{"status":0,"QTime":11},"response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[]},"debug":{"track":{"rid":"...","EXECUTE_QUERY":{"http://url1/|http://url2/":{"QTime":"1","ElapsedTime":"8","RequestPurpose":"GET_TOP_IDS","NumFound":"0","Response":"..."},"http://url3/|http://url4/":{"QTime":"2","ElapsedTime":"8","RequestPurpose":"GET_TOP_IDS","NumFound":"0","Response":"..."}}}}}

after:

{"responseHeader":{"status":0,"QTime":11},"response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[]},"debug":{"track":{"rid":"...","EXECUTE_QUERY":[{"shardAddress":"http://url3","QTime":1,"ElapsedTime":9,"RequestPurpose":"GET_TOP_IDS","NumFound":0,"Response":"..."},{"shardAddress":"http://url2","QTime":0,"ElapsedTime":9,"RequestPurpose":"GET_TOP_IDS","NumFound":0,"Response":"..."}]}}}

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D79578
SOLR-5734 follow-on: make RTimer use System.nanoTime instead of System.currentTimeMillis

Summary: -

Test Plan: -

Reviewers: raiyenga

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D81500
solr: timing-debug to include lucene-search-time (for both ungrouped and grouped searches)

Summary:
Example input/output:

curl "http://localhost:8787/solr/collection1/select?rows=0&wt=json&debug=timing&q=language_id:1"

{"responseHeader":{"status":0,"QTime":69},"response":{"numFound":26,"start":0,"maxScore":0.9354614,"docs":[]},"debug":{"timing":{"time":48.0,"prepare":{"time":0.0,"query":{"time":0.0},"stats":{"time":0.0},"debug":{"time":0.0}},"process":{"time":48.0,"query":{"time":48.0,"LuceneSearchTime":10.0},"stats":{"time":0.0},"debug":{"time":0.0}}}}}

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D81484
solr: add distributedProcess, take, handle, finish sub-stage timers.

Summary:
example output

before:
`{"responseHeader":{"status":0,"QTime":58},"response":{"numFound":627,"start":0,"maxScore":1.0,"docs":[{"suid":"N5Q1JKBE5TS0"}]},`
`"stageTiming":{"time":57.0,`
`"stage1000":{"time":0.0},`
`"stage2000":{"time":47.0},`
`"stage3000":{"time":10.0}}}`

after:
`{"responseHeader":{"status":0,"QTime":62},"response":{"numFound":1275,"start":0,"maxScore":1.0,"docs":[{"suid":"KO0YMS19KNB5"}]},`
`"stageTiming":{"time":60.0,`
`"stage1000":{"time":0.0,"distributedProcess":{"time":0.0},"finish":{"time":0.0}},`
`"stage2000":{"time":51.0,"distributedProcess":{"time":0.0},"take":{"time":46.0},"handle":{"time":0.0},"finish":{"time":0.0}},`
`"stage3000":{"time":9.0,"distributedProcess":{"time":0.0},"take":{"time":9.0},"handle":{"time":0.0},"finish":{"time":0.0}}}}`

Test Plan: -

Reviewers: raiyenga, mjustice, dcollins

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D88065
solr: debug=timing to include metrics surrounding elapsedTime

Summary:
example output

before:
`{"responseHeader":{"status":0,"QTime":64},"response":{"numFound":627,"start":0,"maxScore":1.0,"docs":[{"suid":"N5Q1JKBE5TS0"}]},"debug":{`
`"timing":{"time":33.0,"prepare":{"time":4.0,"query":{"time":3.0},"stats":{"time":0.0},"debug":{"time":0.0}},"process":{"time":29.0,"query":{"time":29.0,"LuceneSearchTime":0.0},"stats":{"time":0.0},"debug":{"time":0.0}}},"submitWaitingTimeSum":3}}`

after:
`{"responseHeader":{"status":0,"QTime":30},"response":{"numFound":627,"start":0,"maxScore":1.0,"docs":[{"suid":"N5PN873M0C8W"}]},"debug":{`
`"distribTiming":{`
`"EXECUTE_QUERY":{"submitWaiting":{"count":4,"sum":3,"min":0,"max":0},"elapsed":{"count":4,"sum":60,"min":15,"max":15},"takeWaiting":{"count":4,"sum":0,"min":0,"max":0}},`
`"GET_FIELDS":{"submitWaiting":{"count":1,"sum":0,"min":0,"max":0},"elapsed":{"count":1,"sum":9,"min":9,"max":9},"takeWaiting":{"count":1,"sum":0,"min":0,"max":0}}},`
`"timing":{"time":12.0,`
`"prepare":{"time":3.0,"query":{"time":3.0},"stats":{"time":0.0},"debug":{"time":0.0}},`
`"process":{"time":9.0,"query":{"time":8.0,"LuceneSearchTime":0.0},"stats":{"time":1.0},"debug":{"time":0.0}}}}}`

Test Plan: -

Reviewers: raiyenga, mjustice, dcollins

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D88641
solr: distribTiming to include debug timing breakdown

Summary:
This is so that min/max/count/sum stats for LuceneSearchTime per-stage are available.

Example output:

  ...
{"responseHeader":{"status":0,"QTime":1541},
"grouped":{ ... },
"debug":{"distribTiming":{

  "TOP_GROUPS":{
"submitWaiting":{"count":4,"sum":2,"min":0,"max":0},
"elapsed":{"count":4,"sum":5353,"min":1265,"max":1388},
"takeWaiting":{"count":4,"sum":0,"min":0,"max":0},
"breakdown":{
  "timing":{"time":{"count":4,"sum":5239,"min":1222,"max":1373},
"prepare":{"time":{"count":4,"sum":20,"min":1,"max":7},"query":{"time":{"count":4,"sum":20,"min":1,"max":7}},"stats":{"time":{"count":4,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":4,"sum":0,"min":0,"max":0}}},
"process":{"time":{"count":4,"sum":5218,"min":1215,"max":1372},"query":{"time":{"count":4,"sum":5218,"min":1215,"max":1372},"LuceneSearchTime":{"count":4,"sum":5128,"min":1195,"max":1350}},"stats":{"time":{"count":4,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":4,"sum":0,"min":0,"max":0}}}}}},

  "EXECUTE_QUERY":{
"submitWaiting":{"count":2,"sum":0,"min":0,"max":0},
"elapsed":{"count":2,"sum":128,"min":56,"max":72},
"takeWaiting":{"count":2,"sum":0,"min":0,"max":0},
"breakdown":{
  "timing":{"time":{"count":2,"sum":103,"min":43,"max":60},
"prepare":{"time":{"count":2,"sum":1,"min":0,"max":1},"query":{"time":{"count":2,"sum":1,"min":0,"max":1}},"stats":{"time":{"count":2,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":2,"sum":0,"min":0,"max":0}}},
"process":{"time":{"count":2,"sum":102,"min":43,"max":59},"query":{"time":{"count":2,"sum":102,"min":43,"max":59},"LuceneSearchTime":{"count":2,"sum":12,"min":6,"max":6}},"stats":{"time":{"count":2,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":2,"sum":0,"min":0,"max":0}}}}}},

  "GET_FIELDS":{
"submitWaiting":{"count":1,"sum":0,"min":0,"max":0},
"elapsed":{"count":1,"sum":41,"min":41,"max":41},
"takeWaiting":{"count":1,"sum":0,"min":0,"max":0},
"breakdown":{
  "timing":{"time":{"count":1,"sum":19,"min":19,"max":19},
"prepare":{"time":{"count":1,"sum":1,"min":1,"max":1},"query":{"time":{"count":1,"sum":1,"min":1,"max":1}},"stats":{"time":{"count":1,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":1,"sum":0,"min":0,"max":0}}},
"process":{"time":{"count":1,"sum":18,"min":18,"max":18},"query":{"time":{"count":1,"sum":18,"min":18,"max":18}},"stats":{"time":{"count":1,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":1,"sum":0,"min":0,"max":0}}}}}}},

  "timing":{"time":5361.0,
"prepare":{"time":22.0,"query":{"time":22.0},"stats":{"time":0.0},"debug":{"time":0.0}},
"process":{"time":5338.0,"query":{"time":5338.0,"LuceneSearchTime":5140.0},"stats":{"time":0.0},"debug":{"time":0.0}}}}}

Test Plan: -

Reviewers: raiyenga

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D95618
solr: distribTiming to include debug timing breakdown (shards.tolerant=true behaviour)

Summary: D95618 added the distribTiming info, for that addition choosing same solution as SOLR-6175 patch.

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D97162
solr: distribTiming timing metrics to optionally include raw values

Summary:
D95618 added distribTiming breakdown itself.

sample output before:

  "elapsed":{"count":4,"sum":599,"min":141,"max":161}

sample output after:

  "elapsed":{"rawValues":[
{"key":"http://nsrchdevnj2:15255/solr/collection1_shard4_replica1/","val":147},
{"key":"http://nsrchdevnj1:15255/solr/collection1_shard3_replica1/","val":144},
{"key":"http://nsrchdevnj2:15250/solr/collection1_shard2_replica1/","val":163},
{"key":"http://nsrchdevnj1:15250/solr/collection1_shard1_replica1","val":213}],
"count":4,"sum":667,"min":144,"max":213}

Test Plan: -

Reviewers: raiyenga

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D99774
Differential Revision: https://all.phab.dev.bloomberg.com/D101206
SOLR-5730: EarlyTerminatingSortingCollector support for chronologically sorted searches

Summary:
 - Merging bbsolr-4.6.1-news EarlyTerminatingSortingCollector support in
 - Changes from Sorter to Sort in various places (bbsolr-4.8.0-news includes LUCENE-5493 Sorter/Sort rename)
 - Tweak SegmentInfo.toString (used for the 'Registered new searcher' trace)
 - Handle removal of mergeSortKey from schema.xml more robustly
   (don't fail requests which continue to send segmentTerminateEarly=true parameter).
 - Adds support for group=true searches, both distributed (CommandHandler.java) and non-distributed (Grouping.java) code paths.

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D78324
Differential Revision: https://all.phab.dev.bloomberg.com/D80718
Differential Revision: https://all.phab.dev.bloomberg.com/D81196
Differential Revision: https://all.phab.dev.bloomberg.com/D80724
lucene: make genLookaheadCountLimit configurable, add genLookbackCountLimit parameter

Summary: DRQS 53627004

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D91140
Differential Revision: https://all.phab.dev.bloomberg.com/D91812
solr: make Lucene's genLook(ahead|back)CountLimit configurable

Summary: DRQS 53627004, D91140 had the correspondingly required lucene change.

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D91776