  Stochastic gradient methods (SGMs) have been widely used for solving
stochastic optimization problems. A majority of existing works assume no
constraints or easy-to-project constraints. In this paper, we consider convex
stochastic optimization problems with expectation constraints. For these
problems, it is often extremely expensive to perform projection onto the
feasible set. Several SGMs in the literature can be applied to solve the
expectation-constrained stochastic problems. We propose a novel primal-dual
type SGM based on the Lagrangian function. Different from existing methods, our
method incorporates an adaptiveness technique to speed up convergence. At each
iteration, our method inquires an unbiased stochastic subgradient of the
Lagrangian function, and then it renews the primal variables by an adaptive-SGM
update and the dual variables by a vanilla-SGM update. We show that the
proposed method has a convergence rate of $O(1/\sqrt{k})$ in terms of the
objective error and the constraint violation. Although the convergence rate is
the same as those of existing SGMs, we observe its significantly faster
convergence than an existing non-adaptive primal-dual SGM and a primal SGM on
solving the Neyman-Pearson classification and quadratically constrained
quadratic programs. Furthermore, we modify the proposed method to solve
convex-concave stochastic minimax problems, for which we perform adaptive-SGM
updates to both primal and dual variables. A convergence rate of
$O(1/\sqrt{k})$ is also established to the modified method for solving minimax
problems in terms of primal-dual gap.
