For light aircraft, weight and its distribution go beyond decorum/comfort to safety of flight.
The aircraft is already fueled by the time the passengers show up, and the pilot has planned on a certain amount, so we don't have that flexibility.