Revert "Revert "Revert "Revert "Revert "Revert "Revert "usb/usb3503: add back the gpio direction setup"""""""

This reverts commit 9e0df29925a938bf87332ae4692ca9513aca7186.