There seems to be nothing wrong with your penis. Underwear doesn't affect growth of penis. The more you think about it, more you will get worried and will have self doubts which is a bigger disease. You can see a urologist in your town.