CORE-5832: extensive refactoring of the job submission endpoint