Don't feel too bad, there are loads of native English speakers who struggle with this very thing. :)