Okay small change to get it to work with a HS device. Apparently gadgetfs doesn't like USB_DT_OTHER_SPEED_CONFIG as the type, even though that is what it should be as far as I can tell. So I am just switching the byte to USB_DT_CONFIG