Before revision:  An expert network is designed in addition to the Q-network and updated each time following the regular offline minibatch update whenever the expert example buffer is not empty  .
After revision:  An expert network is designed in addition to the Q-network , which updates each time following the regular offline minibatch update whenever the expert example buffer is not empty .