Yes, you can overcook the stock. Be sure to follow the recipe and cook it as long as is directed; otherwise, the flavors can taste overcooked.