Yes, gravy does indeed thicken as it cools. Assuming gravy will stay the same thickness as when removed from the heat is a common mistake. If it becomes too thick, you can warm it on the stove and stir in a little bit of chicken stock.