You can spend a great deal of money or very little. You can collect single stamps, sheets, blocks, stamps on envelopes, stamped envelopes, meters on envelopes. You can collect by country, by subject, by date, by region, by anything. You can do all of the above or none of the above.
Most collectors start by trying to collect everything on very little money.
Of course, that's not possible even on very MUCH money. Many of us started by saving the stamps on the envelopes we or our parents received in the mail. Then something would catch our interest and we'd specialize, sometimes in several things at once. Thematic is the largest part of topical collecting, and includes subjects like trains, boats, musical instruments, Presidents, and left-handed midgets. For example, I play trombone on an amateur basis.