It is not necessarily bad, but an empty fridge is less efficient and uses more energy. A full refrigerator or freezer maintains a consistent temperature more easily because the mass of cold foods and drinks inside helps hold the temperature steady.