make panicstr a volatile pointer

this is just to be safe and extra correct rather than to fix any real
issue
add a macro to identify a panic thread

the macro is placed into systm.h as all potential users of this macro
should already include this header file
generic_stop_cpus: move timeout detection code from under DIAGNOSTIC

... and also increase it a bit.
IMO it's better to detect and report the (rather serious) condition and
allow a system to proceed somehow rather than be stuck in an endless
loop.
re-implement hard stopping of CPUs and use it enforce panic(9) context

Hard stopping changes:
- stop_cpus_hard interface is intentionally narrowed to always act on all
  other cpus
- stop_cpus_hard keeps its own accounting of stopped cpus completely
  disregarding soft stopping of cpus (pausing, suspending)
- no recursion whatsoever is allowed for stop_cpus_hard; all callers
  must check if they are already in "super" mode
- protect stop function with spinlock context
- avoid potential deadlock between two cpus already being in nmi context
  (e.g. because of hardware) and trying to stop each other by checking
  the stopped bit while spinning on stopper lock; this is possibly an
  arch-specific thing [x86];

The last item is what I call a cross-cpu deadlock avoidance approach.
General idea: if a CPU can't grab a lock protecting cross-/intra-CPU
interactions, then while spinning on the lock the CPU should check
for incoming cross-CPU events possibly posted by a CPU that currently
holds the lock.  In this scheme an IPI is used only to interrupt a CPU
so that it can notice a new event, but the actual event should be passed
via the memory (a bit in a cpu set - like newly introduced
hard_stopping_cpus, or some other variable).

Panic context changes:
- ensure that only one (panic-ing) CPU runs after panic(9) call by
  stopping other CPUs using stop_cpus_hard()
- disable interrupts/preemption early in panic(9) and do re-enable
  (this most probably breaks sync-on-panic behavior)
- allow a panic-ing thread to penetrate all locks and never sleep on
  conditions (lockmgr locks are exempt actually):
  o mutex, spin mutex, rwlock, rmlock, sx: allow lock/unlock operations to
    unconditionally succeed for a thread in panic [jhb]
  o tsleep, msleep (msleep_spin, _sleep): return success without waiting
    for a thread in panic [jhb]
  o cv code already does that when panisctr != NULL
- boot(): do not rebind to bsp if panic-ing [mdf]
- in all places where we want to stop other cpus first check if they
  are not stopped already

cpu_reset() and cpustop_hard_handler() are only (re-)implemented for
amd64 at the moment.
change generic_stop_cpus to use the same approach hard_stop_cpus
simplify the way of execution of reset on BSP

instead of restarting the BSP and having a special restart hook,
the hook is placed in the spin loop and it is triggered simply
by setting hook function to non-NULL
a style nit in panic()
some whitespace nits
some whitespace nits
a whitespace nit
a whitespace nit
a style nit
increase how long we wait for BSP to become active, amd64 only
do reset on current CPU if somehow BSP doesn't become active, amd64 only
re-implement smp rendezvous code

- create one rendezvous (outgoing) mailbox per each cpu where a cpu would
  place its rendezvous request directed to other cpus
- create a cpu mask for each cpu where other cpus can set a bit to indicate
  that they send a rendezvous request to the cpu in question
- send an ipi only for a first rv request, piggyback subsequent
  requests if a target cpu is still processing previous incoming requests
- many-to-many rv requests can be sent now, there is no locking, the only
  limitation is that a cpu can have only a single outgoing request at
  a time
- to avoid deadlocks, when a cpu waits for its requested to be completed
  by target cpus, it also checks for and processes incoming requests
- to avoid deadlock with cpu stopping logic, cpus also check for stop
  requests while waiting
- there can be only one cpu asking other cpus to stop; this is implemented
  via a handrolled spin mutex analogue; similar to the above, to avoid
  deadlocks a cpu spinning for this lock also checks for an incoming stop
  request
- implement tlb shootdowns via smp rendezvous mechanism, no special ipis
  are needed now, amd64 only (see if the code can be further simplified)
- thus the smp_ipi_mtx is not needed any longer
some whitespace nits
(re-)add another rendezvous counter

The problem was that if a master cpu calls rendezvous in rapid succession
then the rendezvous object could be re-used while some slave cpus were
still spinning on the dual-function teardown/exit counter if a teardown
function was actually specified.
To fix this problem use separate counters, one for teardown entry and the
other for signaling full rendezvous completion.
[fixup] generic_stop_cpus: release stopper_cpu

Mismerged this between git and svn.
smp_rendezvous: correctly support concurrent complex rendezvous

This is a big change.
A problem with the previous approach can be demonstrated by this example:
- processors Ps1 and Ps2 issue complex rendezvous requests R1 and R2
  to processors Pt1 and Pt2
- because of request timings Pt1 gets R1 before R2 while Pt2 gets R2 before R1
- Pt1 executes setup action of R1 and waits for Pt2 to do the same
- Pt2 executes setup action of R2 and waits for Pt1 to do the same
- deadlock

New approach is that we should check for incoming events while waiting
in between of actions of a complex rendezvous event.  In the example above
this means that Pt1 after execution of action of R1 would then notice
R2 and execute its setup action.  Thus we allow actions of different
requests to be interleaved.  The only important limitation is that target
CPUs should not leave rendezvous handling context until all of rendezvous'
actions are executed.
To implement this approach all rendezvous sequence points are coordinated
via a rendezvous' master CPU.  Slave CPUs just execute one action and go
back to main event processing loop.  The master CPU waits for all targeted
slave CPUs to executed the current action (and while waiting it also checks
for incoming events).  When all the slave CPUs finish executing current
action, then the master CPU signals them to execute the next action.

Some implementation details:
- master CPU increases work count for slave CPUs by total count of
  non-no-barrier actions in a rendezvous, so that the slave CPUs
  do not leave rendezvous handling context until all actions are done
- master CPU processes incoming request in between posting its
  actions
- master CPU bumps its work count before posting action, so that
  other CPUs know that the master CPU can process incoming events
  without being IPI-ed
- master CPU processes all incoming events before leaving smp_rendezvous
  so that it fully completes incoming complex rendezvous without
  leaving the context
better name and place for a macro that checks if curthread is panic thread

Suggested by:	attilio