The main downside to eating walnuts is that it's easy to eat too many of them. They are very calorie-dense, so if you're trying to lose weight, eating too much can contribute to weight gain. It's recommended you be mindful of the portion size. Use walnut meat as flavoring, and not the main course.