  We consider stochastic optimization problems which use observed data to
estimate essential characteristics of the random quantities involved. Sample
average approximation (SAA) or empirical (plug-in) estimation are very popular
ways to use data in optimization. It is well known that sample average
optimization suffers from downward bias. We propose to use smooth estimators
rather than empirical ones in optimization problems. We establish consistency
results for the optimal value and the set of optimal solutions of the new
problem formulation. The performance of the proposed approach is compared to
SAA theoretically and numerically. We analyze the bias of the new problems and
identify sufficient conditions for ensuring less biased estimation of the
optimal value of the true problem. At the same time, the error of the new
estimator remains controlled. We show that those conditions are satisfied for
many popular statistical problems such as regression models, classification
problems, and optimization problems with Average (Conditional) Value-at-Risk.
We have observed that smoothing the least-squares objective in a regression
problem by a normal kernel leads to a ridge regression. Our numerical
experience shows that the new estimators frequently exhibit also smaller
variance and smaller mean-square error than those of SAA.
