Several stamp catalogs exist. Once you collection is underway, you may want to get a lot more stamps. Stamp stores and mail-order firms sell stamps in two basic ways. One is in packets which means the stamps are already off envelope paper and typically sorted for you. These are usually sold by country.
Packets tend to be expensive.
The other way they are sold is by volume on paper. This means that you are getting a big chunk of stamps still attached to small fragments of an envelope for you to soak off. These are much, much cheaper and you can sometimes get thousands of stamps. As always, be cautions with any dealings with a mail order company until you know they are trustworthy.