Fix case where port is missing

Backport of https://github.com/owncloud/core/pull/7386 to stable5

Please review/test @karlitschek @LukasReschke @schiesbn