Every number is used as the base of one of the columns.
Of course, with this system you need to keep inventing new digits to write larger numbers. On the other hand, with just digits 0-9 you can express numbers from 0 to 3628799. It isn't inconceivable that the factorial system could have been used by early cultures, but I don't know of such a case. The factorial system is somewhat inconvenient for computation, but no more than, say, Roman numerals. This places some restrictions on our possible number systems.