in case that loading from the cache fails, load from wkhtmltopdf without
cache using the user agent string given in the crawl profile
show number of threads on status page
set Busy- and Blocking-Threads to daemon mode (they will now not prevent
YaCy from termination if still running)
showing list of all thread in threaddump using the ThreadMXBean counter
(this obviously show more threads than before?)
Replaced all fixed thread pools with cached thread pools. The cached
thread pools will flush their cached (dead) threads after 60 seconds.
This will cause that YaCy now runs constantly withl about 50 threads,
about 100 at peak times. Previously, about 400 threads had been cached
and kept in a hibernation state, which caused that the numproc counter
in /proc/user_beancounters (exists only in VM-hosted linux) was as high
as the cached number of threads. This caused that VM supervisors
terminated whole VM sessions if a limit was reached. Many VM providers
have limits of numproc=96 which made it virtually impossible to run YaCy
on such machines. With this change, it will be possible to run many YaCy
instances even on VM hosts.