Avoid wearing plaids or other patterns that are too busy with a flannel shirt, as it can create a chaotic and distracting look.