Revert "Revert "Revert "Revert "usb/usb3503: add back the gpio direction setup""""

This reverts commit 906e57f2421e517f9c3fa8c6ba6da96efa48f6ca.