So if you don't wash your face daily, you can start to get a mild infection in the pores of your skin. For most people, this isn't a particularly big deal, but it does tend to make your skin itchy, flaky, or inflamed. Just wash your face every day and this problem should go away.