It depends on your child’s nap schedule.
If they still take two naps a day, try to time the party between them. This would roughly be around noon.
If they are taking one nap, time the party before or after their nap. Guests might not like the sound of a party starting at 9 a.m., so 2 p.m. may suit them better.