The best way to stop your hoodie from being rough is to stop frequently washing it, as it is harsh on the fibres.
Whenever you do wash them, turn them inside out, cold wash them and make sure they air dry. Unlike regular hoodies, the SA1NT hoodies are 380GSM and are a brushed 60/40 cotton/poly fleece so you can rest easy and know that it will get rough.