Yes. After the flavors from the spices infuse the soup, you could take them out of the dish before serving. Leaving the clove is fine too as it is edible.