In baking, there’s no real downside to using almond flour. Your baked goods will just cook a little quicker and have a slightly different texture and consistency. The only real downside compared to regular flour is the difference in cost. Almond flour tends to be more expensive by weight.