You only need the people who are involved in whatever people are under discussion. Those who don't want or need to be there are more productive elsewhere. Keep group focused on the topic . Don't get caught without something important or leave everyone waiting for someone to roll in an audiovisual cart. Otherwise you are left trying to fill time and your attendees must try to feign enthusiasm.
People hate waiting for meetings to start and hanging around waiting for the last speaker to stop.
A well-made agenda can keep this problem to a minimum. Try to move quickly from point to point so attendees aren't lost in endless talk. Keep disputes to a slow boil. If the conversation gets too hot, let each side make its point and then move on or close the meeting. You don't need to have your company's controversies and dirty laundry aired out before a large group.