Fix the standard port that is inferred from url scheme

This is a port of this review (https://review.openstack.org/#/c/39210/2)
to try and cleanup and old bug.

Change-Id: Idf93053c9f4a49f65d9d52e879a88d79c9fd3170
Co-Authored-by: Martin H Berwanger <martin.berwanger@gmail.com>
Fixes: bug #1205317