A passport holder is designed specifically to hold your passport, while a travel wallet is designed to hold all of your important travel documents such as your passport, boarding pass, credit cards, and cash. A travel wallet typically has more compartments and pockets than a passport holder.