* Histogram ring-buffer of metrics for this plugin.
* Using pearson correlation + histogram data for the plugin to detect a
 negative correlation between pool size and duration, which lets us cap
 the pool size at an appropriate limit for the hardware you're using.
* Better calculations / modeling for workload estimation and required threads, using a duration sort and historic data to imply future performance.
* Thread pools scale fairly agressively to fulfill target (currently 1/2 the interval) collection timeframe, and based on correlation feedback will
cap at an appropriate level (apparently).
* Better effort estimation using the average time for all tasks for unknown tasks.
* Better detection of correlation change -- direction change should probably 'dynamic' rather than targeting a set correlation number. But this is a good start to prove the idea.
* Started using commons-math in an effort to see if I could use a interpolation to a polynomial univariate function in order to find the local minimum of a polynomial function fit to histogram data points.

This commit includes those compuatations and some basic force-fitting, I'll clean this up in a future commit, but don't want to loose the progress, since it seems fairly accurate at this point.
* Initial commit of working predictive pool sizing.

* TODO: Configuration for target latency, histogram, etc.
Changed the optimalFound to optimalSearch indicating tri-state, search for a smaller pool, found optimal, or search for a bigger pool.
Committing a few changes here...

* Slightly better pool size clamping. It never goes over the number of data points to collect, or the MAXIMUM_THREADS, whichever is the lower limit.

* I'm still not 'thrilled' with the way the polynomial interpolation is going... I may add more datapoints for consideration.
Better initialization.
now.
Don't blow up on reading NULLs. NULL is a legit value in java. I just wish I could get the plugin to respect that I sent it the -absense- of a value... oh wait. I should just not submit anything... :-p
Don't blow up on reading NULLs. NULL is a legit value in java. I just wish I could get the plugin to respect that I sent it the -absense- of a value... oh wait. I should just not submit anything... :-p
Implemented LeastSquares projection based on weighted analysis of a histogram containing recent collections.

This is a major reorganization of the code from a separation of concerns and implemenation standpoint. There's still a lot of 'polish' to do, but at this point I'm pretty pleased with the results.

The basic premise is:
When the plugin starts up it uses a fibonacci sequence to size the thread pool handling collections. It requires 1/2 the size of the histogram (currently 15) to do a 'projection' where it plots the average durations for each pool size it's executed, and weights the point based on the % of attributes which were not cancelled + % of the collection latency for the read cycle.

Based on my own observations on my own hardware, with hand-tuning things over the last few weeks, the projection is really really close to optimal -- certainly within a thread or two, and sufficient enough for my purposes at this time.

The thing I'm not doing yet is weighting based on workload deviation from the most recent read cycle, which I should (and certainly could) add in.
Updated the javadoc on this
* Field protections in place.
* Better reset / backoff of the fibonacci sequence.
Fix the fibonacci sequence resets. Things are simply too agressive.
More resiliant to network hiccups, now takes two cycles of increasing cancellations.
It's more verbose right now, but the fib tweak is less of a freak
* General cleanup (unused methods, format)
* Move Collectd.dispatch to the main read() loop thread.
* Better use of timing / interval in the dispatched values.
* Log level tweaking....
* Internal metric collection (currently reported as localhost -- Collectd.getHostname() is causing issues...)
* Tuning / updates for scoping and what-not.
* Interval is always internally a millisecond value.
If we hit maxThreads, clear and try to find a better solution.
* Added CollectInternal, MaxThreads configuration
* Modified internal collection methods.
* Only recalculate when the pool _grows_.