Before revision:  In the general noisy case, almost sure convergence of the methods fitting into the framework is proved when line searches and suitably decaying step lengths are combined. When the objective function is a finite sum, such as in machine learning applications, our framework is specialized as a stochastic L-BFGS method with line search only, with almost sure convergence to the solution. In this case, linear convergence rate of the expected function error is also proved , along with a worst-case \mathcal{O (\log(\varepsilon^{-1})) complexity bound} .  Numerical experiments, including comparisons with state-of-the art first- and second-order stochastic optimization methods, show the efficiency of our approach.
After revision:  In the general noisy case, almost sure convergence of the methods fitting into the framework is proved when line searches and suitably decaying step lengths (\log(\varepsilon^{-1})) complexity bound} . Numerical experiments, including comparisons with state-of-the art first- and second-order stochastic optimization methods, show the efficiency of our approach.