No, gems, much like shards, are scattered around the world, although in marginally lower amounts. They cannot, however, be bought by any of the game vendors for an unlimited amount.