  We propose a novel numerical method for high dimensional
Hamilton--Jacobi--Bellman (HJB) type elliptic partial differential equations
(PDEs). The HJB PDEs, reformulated as optimal control problems, are tackled by
the actor-critic framework inspired by reinforcement learning, based on neural
network parametrization of the value and control functions. Within the
actor-critic framework, we employ a policy gradient approach to improve the
control, while for the value function, we derive a variance reduced least
square temporal difference method (VR-LSTD) using stochastic calculus. To
numerically discretize the stochastic control problem, we employ an adaptive
stepsize scheme to improve the accuracy near the domain boundary. Numerical
examples up to $20$ spatial dimensions including the linear quadratic
regulators, the stochastic Van der Pol oscillators, and the diffusive Eikonal
equations are presented to validate the effectiveness of our proposed method.
