Revert "Revert "Revert "Revert "Revert "Revert "Revert "Revert "Revert "usb/usb3503: add back the gpio direction setup"""""""""

This reverts commit 7320ff1f67272716fa5bc6cd9a1bd9afd255e3fd.