[v6r12] Fixes for the new pilot

Some real bugs, some more documentation, some slight code changes mostly for logging, debugging, and extensibility
This was done deliberately to avoid double headers - from JobAgent and from the local logger. In the pilot using logger is not as crucial as elsewhere