dgrnbrg/autotest — comment on commit bd60399577cf89525b8ff2b40c62c977a319371c
File: server/hosts/ssh_host.py line 280
By dgrnbrg:

Unfortunately, killing remote processes is impossible to do reliably, so we must decide what the best approach is for the situation.

I designed this for the scenario that you want to spawn long-running daemons and provide an absolute maximum time for every process to run to avoid a hung test.

One solution to the deadlock you observed is to ensure that wait_for() always gets a timeout if it's a remote AysncJob.

As for the noisy message from the "kill" command, it'll be difficult to avoid that. This API is meant to interface with existing daemons that must be tested, and many of those daemons already have very noisy stdout and stderr, so you should be able to use the SortingLogger that was recently merged into AutoTest to put those error messages into the appropriate place if needed.

Would that change and explanation of the behavior address your concerns?

I agree that it's not ideal that the "line 0: kill: (xxx) - No such process" is bad, but that little shell-script hack is the only way to kill a process started on a remote machine safely, since there's no way to get the PID of the remote process without running a command in the shell that starts the process, and a killall would potentially impact the other parts of the system.