solr: debug=timing to include metrics surrounding elapsedTime

Summary:
example output

before:
`{"responseHeader":{"status":0,"QTime":64},"response":{"numFound":627,"start":0,"maxScore":1.0,"docs":[{"suid":"N5Q1JKBE5TS0"}]},"debug":{`
`"timing":{"time":33.0,"prepare":{"time":4.0,"query":{"time":3.0},"stats":{"time":0.0},"debug":{"time":0.0}},"process":{"time":29.0,"query":{"time":29.0,"LuceneSearchTime":0.0},"stats":{"time":0.0},"debug":{"time":0.0}}},"submitWaitingTimeSum":3}}`

after:
`{"responseHeader":{"status":0,"QTime":30},"response":{"numFound":627,"start":0,"maxScore":1.0,"docs":[{"suid":"N5PN873M0C8W"}]},"debug":{`
`"distribTiming":{`
`"EXECUTE_QUERY":{"submitWaiting":{"count":4,"sum":3,"min":0,"max":0},"elapsed":{"count":4,"sum":60,"min":15,"max":15},"takeWaiting":{"count":4,"sum":0,"min":0,"max":0}},`
`"GET_FIELDS":{"submitWaiting":{"count":1,"sum":0,"min":0,"max":0},"elapsed":{"count":1,"sum":9,"min":9,"max":9},"takeWaiting":{"count":1,"sum":0,"min":0,"max":0}}},`
`"timing":{"time":12.0,`
`"prepare":{"time":3.0,"query":{"time":3.0},"stats":{"time":0.0},"debug":{"time":0.0}},`
`"process":{"time":9.0,"query":{"time":8.0,"LuceneSearchTime":0.0},"stats":{"time":1.0},"debug":{"time":0.0}}}}}`

Test Plan: -

Reviewers: raiyenga, mjustice, dcollins

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D88641
lucene: make genLookaheadCountLimit configurable, add genLookbackCountLimit parameter

Summary: DRQS 53627004

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D91140
lucene: getGenLook(ahead|back)CountLimit needs considering in TestIndexWriterConfig

Summary: D91140 has the original lucene/core-only changes.

Test Plan: `ant test -Dtestcase=TestIndexWriterConfig` passes.

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D91812
solr: make Lucene's genLook(ahead|back)CountLimit configurable

Summary: DRQS 53627004, D91140 had the correspondingly required lucene change.

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D91776
solr: add SolrConfig/updateHandler/indexWriter/closeWaitsForMerges=FALSE support - bug fix (affects bbsolr-4.8.1-news only)

Summary:
Previous patch (D84911) resulted in DirectUpdateHandler2 calling close(boolean) instead of close() on its writer variable, and because IndexWriter has a close(boolean) method but SolrIndexWriter (which inherits from IndexWriter) only has a close() method this meant that formerly the inherited class' close() was called but now the base class' close(boolean) was called. This fix fixes that.

https://issues.apache.org/jira/i#browse/SOLR-6125 ticket and patches unaffected since in all those places the close(boolean) method no longer exists and there is instead a waitForMerges() method.

Test Plan: Tested okay on dev by before-and-after log inspection.

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D92552
lucene: add SolrConfig/updateHandler/indexWriter/closeWaitsForMerges=FALSE support - bug fix (affects bbsolr-4.8.1-news only)

Summary:
Follow-on from D92552 patch which added a SolrIndexWriter.close(boolean) method.

In order to avoid a call chain like this:

  Something() calls
  SolrIndexWriter.close() calls
  SolrIndexWriter.close_impl(null) calls
  IndexWriter.close() calls
  SolrIndexWriter.close(boolean)
  SolrIndexWriter.close_impl(boolean)
  IndexWriter.close(boolean)

we must introduce a IndexWriter.close_impl(boolean) to get a call chain like this:

  Something() calls
  SolrIndexWriter.close() calls
  SolrIndexWriter.close_impl(null) calls
  IndexWriter.close() calls
  IndexWriter.close_impl(...)

Test Plan: `ant test -Dtestcase=TestCoreContainer` now passes again.

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: dcollins

Differential Revision: https://all.phab.dev.bloomberg.com/D93282
solr: add shutdownCoresCloseTimeoutSeconds support with corresponding TestCoreContainer test

Summary:
In DRQS 53627004 error-in-final-commit during solr-shutdown led to corrupted segments_.... file which prevented subsequent solr-start.

This change (subject to a timeout) delays solr-shutdown until all requests have stopped using all solr cores. It also prints any suppressed exceptions within the error-in-final-commit exception.

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D92562
solr: distribTiming to include debug timing breakdown

Summary:
This is so that min/max/count/sum stats for LuceneSearchTime per-stage are available.

Example output:

  ...
{"responseHeader":{"status":0,"QTime":1541},
"grouped":{ ... },
"debug":{"distribTiming":{

  "TOP_GROUPS":{
"submitWaiting":{"count":4,"sum":2,"min":0,"max":0},
"elapsed":{"count":4,"sum":5353,"min":1265,"max":1388},
"takeWaiting":{"count":4,"sum":0,"min":0,"max":0},
"breakdown":{
  "timing":{"time":{"count":4,"sum":5239,"min":1222,"max":1373},
"prepare":{"time":{"count":4,"sum":20,"min":1,"max":7},"query":{"time":{"count":4,"sum":20,"min":1,"max":7}},"stats":{"time":{"count":4,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":4,"sum":0,"min":0,"max":0}}},
"process":{"time":{"count":4,"sum":5218,"min":1215,"max":1372},"query":{"time":{"count":4,"sum":5218,"min":1215,"max":1372},"LuceneSearchTime":{"count":4,"sum":5128,"min":1195,"max":1350}},"stats":{"time":{"count":4,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":4,"sum":0,"min":0,"max":0}}}}}},

  "EXECUTE_QUERY":{
"submitWaiting":{"count":2,"sum":0,"min":0,"max":0},
"elapsed":{"count":2,"sum":128,"min":56,"max":72},
"takeWaiting":{"count":2,"sum":0,"min":0,"max":0},
"breakdown":{
  "timing":{"time":{"count":2,"sum":103,"min":43,"max":60},
"prepare":{"time":{"count":2,"sum":1,"min":0,"max":1},"query":{"time":{"count":2,"sum":1,"min":0,"max":1}},"stats":{"time":{"count":2,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":2,"sum":0,"min":0,"max":0}}},
"process":{"time":{"count":2,"sum":102,"min":43,"max":59},"query":{"time":{"count":2,"sum":102,"min":43,"max":59},"LuceneSearchTime":{"count":2,"sum":12,"min":6,"max":6}},"stats":{"time":{"count":2,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":2,"sum":0,"min":0,"max":0}}}}}},

  "GET_FIELDS":{
"submitWaiting":{"count":1,"sum":0,"min":0,"max":0},
"elapsed":{"count":1,"sum":41,"min":41,"max":41},
"takeWaiting":{"count":1,"sum":0,"min":0,"max":0},
"breakdown":{
  "timing":{"time":{"count":1,"sum":19,"min":19,"max":19},
"prepare":{"time":{"count":1,"sum":1,"min":1,"max":1},"query":{"time":{"count":1,"sum":1,"min":1,"max":1}},"stats":{"time":{"count":1,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":1,"sum":0,"min":0,"max":0}}},
"process":{"time":{"count":1,"sum":18,"min":18,"max":18},"query":{"time":{"count":1,"sum":18,"min":18,"max":18}},"stats":{"time":{"count":1,"sum":0,"min":0,"max":0}},"debug":{"time":{"count":1,"sum":0,"min":0,"max":0}}}}}}},

  "timing":{"time":5361.0,
"prepare":{"time":22.0,"query":{"time":22.0},"stats":{"time":0.0},"debug":{"time":0.0}},
"process":{"time":5338.0,"query":{"time":5338.0,"LuceneSearchTime":5140.0},"stats":{"time":0.0},"debug":{"time":0.0}}}}}

Test Plan: -

Reviewers: raiyenga

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D95618
solr: cherry-picking SOLR-6175

Summary:
https://issues.apache.org/jira/i#browse/SOLR-6175
SOLR-6175: DebugComponent throws NPE on shard exceptions when using shards.tolerant
SOLR-6175: Fix failing test by removing assert that matches exception message. They might be different between platforms.
SOLR-6175: Assert that exception message is not null

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D97146
solr: distribTiming to include debug timing breakdown (shards.tolerant=true behaviour)

Summary: D95618 added the distribTiming info, for that addition choosing same solution as SOLR-6175 patch.

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D97162
TREQ 2799103: solr: Start RTimer for SearchHandler from right when the request starts

Summary: Snapshot start time as soon as we get control from Jetty, to get a more accurate measure.

Test Plan: Tested on Dev

Reviewers: cpoersch, dcollins, mjustice

Reviewed By: cpoersch

Differential Revision: https://all.phab.dev.bloomberg.com/D97800
solr: cherry-pick SOLR-6223

Summary: SOLR-6223: SearchComponents may throw NPE when using shards.tolerant and there is a failure in the 'GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG' phase

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice, ksrivisa, mwashimi

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D98590
solr: add admin/collections?action=DELETEREPLICA&core=... support

Summary: -

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D98798
Revert "solr: add admin/collections?action=DELETEREPLICA&core=... support"

Summary: This reverts commit 4a491aca27763540688d0b593b5dcebdf8ec6a30.

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D99186
solr: add admin/collections?action=DELETEREPLICA&core=... support, make collection=... and shard=... parameters optional

Summary: -

Test Plan: -

Reviewers: raiyenga, dcollins, mjustice

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D99184
solr: distribTiming timing metrics to optionally include raw values

Summary:
D95618 added distribTiming breakdown itself.

sample output before:

  "elapsed":{"count":4,"sum":599,"min":141,"max":161}

sample output after:

  "elapsed":{"rawValues":[
{"key":"http://nsrchdevnj2:15255/solr/collection1_shard4_replica1/","val":147},
{"key":"http://nsrchdevnj1:15255/solr/collection1_shard3_replica1/","val":144},
{"key":"http://nsrchdevnj2:15250/solr/collection1_shard2_replica1/","val":163},
{"key":"http://nsrchdevnj1:15250/solr/collection1_shard1_replica1","val":213}],
"count":4,"sum":667,"min":144,"max":213}

Test Plan: -

Reviewers: raiyenga

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D99774
solr: distribTiming timing metrics to optionally include raw values - fix

Summary: D99774 follow-on

Test Plan: -

Reviewers: raiyenga

Reviewed By: raiyenga

Differential Revision: https://all.phab.dev.bloomberg.com/D101206
solr: Log params used for UpdateShardHandler HTTP client

Summary: Currently there's no way to figure out what params are being used.

Test Plan: Untested, staged rollout

Reviewers: cpoersch

Reviewed By: cpoersch

Differential Revision: https://all.phab.dev.bloomberg.com/D101724
SOLR-6136: Fix spin lock in blockUntilFinished (which eats CPU unnecessarily) using wait/notify and add a unit test for ConcurrentUpdateSolrServer

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1610864 13f79535-47bb-0310-9956-ffa450edef68
SOLR-6136: backport fixes for precommit issues

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1610908 13f79535-47bb-0310-9956-ffa450edef68