Ensembles/ert — pull request #204 by chflo (2013-10-01 15:00:20+00:00)
Title: Stop simulations

Stop simulations:

Implemented monitoring of the runtime of simulations; kill the jobs which takes to long time to run.
The decision on when to stop the simulation can be set as a MIN_REALIZATIONS keyword together with a STOP_LONG_RUNNING keyword, or a MAX_RUNTIME keyword.

If new config keyword STOP_LONG_RUNNING is set to TRUE, the keyword MIN_REALIZATIONS must be set. The jobs left in the job_queue when MIN_REALIZATIONS have run successfully are killed after a time.
The time from MIN_REALIZATIONS have run to killing of the rest of the jobs is set to 25 % of the average runtime for successfull jobs.

If new config keyword MAX_RUNTIME is set, each job in the queue is only allowed to run in MAX_RUNTIME (given in seconds).

If both STOP_LONG_RUNNING + MIN_REALIZATIONS and MAX_RUNTIME is specified,
the jobs are killed on first occurence of these timers.

Also done some cleanup of memory leaks