opened
oxinabox
username_0
Its long been my opinion (with little supporting evedance) that  hyper-parameter optimization 
is just a special case of heuristic global optimization.
The world that simulated anealling, particle swarm optimization, and genetic algorithms dominate have been in for ages.

For lower dimentions bayes-opt is likely much better,
but once you start getting to dozens, or hundreds of hyper-parrameters,
it seems worth exploring these gradient free methods.

How hard is it to use gradient-free methods from Optim or BlackBackOptim with this?
Integration of BlackBoxOptim / Gradient-Free Optim methods?
issue
created
baggepinnen
username_1
I support your thesis, hyperopt is essentially global optimization. This package is mostly written to supply the macro, which facilitates a nice syntax that makes it easy to slap optimization of hyperparameters onto already existing code (I'm often a cowboy coder). Internally, the interface is far from optimally designed, as I initially did not intend to support much more than random search (random search is a very strong and hard-to-beat baseline). I have since added a number of methods without fundamentally redesigning the internals. Thus unfortunately makes it a bit awkward to add support for new methods 😔
comment
created
oxinabox
username_0
we can just leave this open til that day.
comment