It's expensive in terms of complexity, management and non-ease of use. It's also expensive in terms of the processing power required. But attempts to minimize the impact of the latter on Web servers are now affecting the very security that is being sought.
Web security is pretty good stuff.
The basic idea is that you want to set up secure communication between yourself and a server that you are sure is the right one. At the same time, the server wants to be sure that it's talking to you and not someone down the street. The server has a digital certificate your browser uses to be sure it is communicating with the right server. Thus, the server knows who you are. VPNs can be used between firewalls, in which case the communication inside the firewalls is open to eavesdropping, and the identity of the servers and users is not assured.