ssh: fix strange connection related issues

Signed-off-by: Luka Perkov <luka.perkov@sartura.hr>