Create an active executor monitor that will track pending executors