Yes, rice pudding will thicken as it cools as the rice continues to absorb the milk.