For children under 8 years old, invite as many children as the child's age. Some parents choose to add 1 or 2 to this equation. Only you know your limits. Of course as your child gets much older the formula will need revision. Most parents enjoy the break This way you won't feel so much pressure to entertain Adults.
Competitive games are enjoyable only for the people who win.
Avoid tears by selecting games that everyone gets a chance to succeed. If it's a game like Musical chairs, where some one is out, make them the leader of the music. Then let the player join back in after one round. Better yet, plan a few cooperative games, in which the whole gang faces a challenge, such as following a treasure map, then they each receive a prize or split the treasure evenly. Try setting up your party in activity stations to lead the children in the direction of the party focus.