Is "In addition to limiting the total average power used by the device, the USB specification limits the inrush current (i.e., that used to charge decoupling and filter capacitors) when the device is first connected." a good answer to the question "When is the inrush current affected by the USB specification?"
yes