We will protect what you need in order to do your work on the understanding that you won't do positive harm to society at large. I think shield laws are a good example of this contract. Under these laws, journalists cannot be forced to reveal confidential sources. There's a strong parallel between journalists and lawyers in terms of the obligation to protect confidences. Journalists will go to jail before they will breach a promise to a source that they will keep the source's identity confidential because they believe, once it's perceived their promise won't be kept, the source of news that ultimately serves the public good will just dry up.
Lawyers see it from the same perspective.
The promise of confidentiality serves the need for the lawyer to have a relationship of candor with the client. You can tell me anything, and it won't get beyond me. And you can have confidence that when you tell me something, I'm not going to use it against you. In general, I think one of the problems is the extent to which there is public participation in formulating rules of professional conduct. The legal profession has moved in the direction of public participation, at least to the point of having public representatives on the governing board of the bar.