> If the goal is to maximize overall (combined consumer and producer) surplus

The goal is to maximise wealth in the whole economy in such a way that gains incurred by one agent cannot come from harm incurred by another (Pareto optimality).

> My entire point is that it isn't possible for everyone to be as happy as they could be in a purely distributive scenario. If there is $3 of surplus on the table then whoever doesn't get all of it is going to be less happy than if they did.

It's not about surplus!

Take a good X and denote it's cost C(X). Assume we're using pennies (i.e only dealing with real, positive integers) and that the market is large and normally distributed. For ANY flat price, P(X) = c (a constant), such that P(x) > C(X)+1 there must exist a class of consumer whose true monetary valuations are in the set V = { v | C(X) < v < P(x) }. They will never be catered for and are always worse off than they could be. Consumers with v > P(x) are gaining v - P(X) at the expense of: the producer, who is incurring an opportunity cost of v - P(X); and members of V, who are each losing Utility(X) - v. We therefore have utility deltas vs. price discrimination of (assuming no indifference for simplicity):

u1 = Utility1( Consumer1 w/ v1 > P(X) ) = U1( v1 - P(X) ) > 0

u2 = U2( Consumer2 w/ v2 in V ) = U2(v2) - U2(X) < 0

u3 = U3( Producer ) = -U3( v1 - P(X) ) - U3(v2) << 0

For this to have been fair (Pareto) to everyone, these deltas must be u1 = u2 = u3 = 0. As it stands, the inefficient pricing structure delivers | u1 | < | u2 + u3 | i.e. wealth lost by two is greater than wealth gained by one - wealth is destroyed. Violation of Pareto is obvious here, but showing the wealth destruction rigorously is a little more work than I'm willing to do in ASCII, but it's there.

Solutions? Well the producer could set a flat price P(X) = C(X), essentially giving all their surplus to the consumers. This removes profit from the equation, which under imperfect competition removes the incentive for the producer to go on doing anything at all, and nullifies the analysis (as well as being completely unrealistic.) The only way to make sure everyone is as happy as they can be without harming anyone else (i.e. members of V) is to make P(X) = vn for any given consumer n.

You may not like the idea of producers taking all the potential consumer surplus, but in the absence of perfect competition, this surplus can only ever be consumer surplus at the detriment of other consumers (members of V) and the producer.

> But you're also ignoring the point about competition.

Perfect competition and perfect price discrimination are equivalent under the usual perfect assumptions, you're quite right. But when we recognise that neither is actually possible this becomes less relevant.

Perfect competition always results in P(X) = C(X) and this whole discussion is moot. Imperfect competition (in the absence of government subsidies, etc) results in P(X) > C(X). Again, members of V are missing out and the case for price discrimination is created. The strength of the case for price discrimination is a function of the sum of the members of V, which in turn is a function of [P(X) - C(X)] assuming consumer valuations are distributed somewhat normally. Therefore, one could argue that the "less perfect" the competitive environment is, the stronger the case for price discrimination (I need not point out that the current state of the world is far from economically perfect). In fact - under the assumption that perfect competition and perfect price discrimination are not possible - I suspect it could be shown that an economy implementing some "good-enough" version of both is optimal.