Before revision:  An expert network is designed in addition to the Q-network and updated each time following the regular offline minibatch update whenever the expert example buffer is not empty  .
After revision:  An expert network is designed in addition to the Q-network and updated each time following the regular offline minibatch update whenever the expert example buffer is not empty . The Q-network plays the role of the advantage function only during the update .