Yes, the batter should be lumpy. It's important to not over mix the batter because those lumps help make the muffins fluffy.