I'm not convinced this explanation-by-analogy style is really all that helpful, particularly with regards to "the big bang is an expanding balloon". What makes it such a confusing analogy is that most people were already picturing something like an expanding sphere anyway, just one filled uniformly with matter moving away from a central point as it expands. And it doesn't reinforce that the Universe has no center, since most people can easily imagine what the center of a balloon is.

So, screw the analogy. Here's what we know.

Right now, everything is moving away from us in proportion to its distance from us. Read that again, because that's the fundamental observation. The usual intuitive view that lay people have is that we're drifting away from the Big Bang on inertia left over from the explosion. This would not result in what we see. For everything to move away from us in direct proportion to its distance, everything would have to be speeding up all of the time. When we're distance d from the Big Bang, we're drifting away from it at half the speed that we are when we're distance 2d away. Inertia doesn't do that. So much for inertia.

Maybe it's not inertia, but some kind of Magic Inertia that we don't understand? A kind that pushes things faster and faster as time goes on, like little angels perpetually flapping their wings to add momentum to every atom. That's great for a while, but you hit another problem: the speed of light. If you're already moving away at (nearly) light-speed, you can't double your speed. Doubling your momentum will just give bring you infinitesimally closer to light-speed. D'oh.

So what is the answer? The closest you can get without worrying about math is probably this: Space is being created, and distance is somehow being inserted in the gaps between everything, and this happens continuously and everywhere. The other, more accurate, way to see it is that the meaning of "distance" and "duration" (that is, the metric we use to determine the distance between points, both in space and time) is itself changing as the Universe expands.

This results in some observables that cannot be explained otherwise, and which we do actually observe. In this model, everything (at intergalactic distances) does indeed increase its distance from everything else, and in direct proportion to the original distance. Since this is a change in the spacetime metric, rather than actual movement within spacetime, it does not result in relativistic effects that you would see if it were literal movement. There are galaxies in the sky right now whose distance from us is literally increasing at a rate that would be impossible if it were due to movement, since it would require movement faster than the speed of light. That's the clincher--the reason no other model can possibly work.

This "metric expansion of spacetime" maybe sounds like a cop-out, like something's been invented just to explain the Big Bang, but it hasn't. This is exactly what matter does in the normal scales of planets and stars. The changing of the spacetime metric is what gravity fundamentally is, as Einstein explained with General Relativity. It's the reason clocks run slower on Earth than they do in space, and it's the reason the planets stay in their orbits. It's all really just quite ordinary.