This usage arises from the interesting fact that the acceleration is the same no matter how large the falling object. In other words, a piano falls to the ground just as fast as an egg. You might expect larger objects to fall faster, but Newton's second law of motion says that larger masses actually require a greater force to accelerate them at the same rate. Because there's a greater gravitational force between a large mass and the earth. It just so happens that this extra force exactly matches the extra force needed to move the larger mass.
Large objects require more force to move, but they experience a greater attraction to the earth.
Smaller objects aren't attracted to the earth as much, but they don't require as much force. Thus, all objects fall at the same rate. You can see this fact even more clearly by looking at the mathematical results. Let's look at the case of a banana. Recall that Newton's first law of motion says a banana at rest tends to stay at rest, and a banana in motion tends to stay in motion.