That essay is really rather interesting and makes some great points. I do think that it misses some important areas though. Given that it was written in 2004, I'll give Paul Graham the benefit of the doubt. Most of my points have only really surfaced in the last few years.

1. He states at one point that "It's absolute poverty you want to avoid, not relative poverty.", which is certainly true to an extent, but not completely. A recent BBC documentary looked at some of the poorest people in America.[1] The thing that really struck me about that is that these people, in absolute terms, are not poverty stricken. They're not short of shelter, food or water.
In relative terms though, they've been left behind. The modern american society simply isn't designed for people living on their income level. They live in motels without kitchens, so the option of cooking the basic staple foods eaten by those on similar income levels elsewhere is unavailable to them. They have to buy expensive packaged foods. The high cost of the society they live in means that their disposable income, and their ability to climb out of the economic hole that they're in, is almost certainly less than that of people with their absolute income in a poorer society. Poverty has to be considered in both relative and absolute terms. The people depicted in that documentary have fewer options available to them than those in poor countries. American health costs, education costs and rents are all designed for the average American income level.

[1]: http://www.bbc.co.uk/panorama/hi/front_page/newsid_9694000/9...

2. There are aspects of society that set their own value. I know it's popular to complain about bankers right now, but they're probably the best example of this. Banking a necessary evil to allow us to deal with the vast, money based economy we've created. While almost every other industry operates by trading goods or non-monetary services for money (and so only sees money that's in some way related to their business expenses), banking sees all the money. A "normal" business can increase profits by maintaining or increasing income and reducing expenses (eg outsourcing, manufacturing in China, reducing material costs). Banks, by virtue of their operation, handle incredible amounts of money. They're in a unique situation that allows them to set the value of the service they provide. Not surprisingly, that service value is set considerably higher than perhaps it truly is.

It's obviously hard to place a value on the services provided by banks. In his essay, Paul Graham talks about wealth creation, giving a farmer as an example. The banking (and financial trading system) is not primarily a wealth creator. It's a system for moving that created wealth around. Banking can enable wealth creation with funding, but the system we have today has developed to reward activities with high returns over short periods. What I'd argue as true human wealth creation - manufacturing and valuable service provision - doesn't provide the returns that the banking system was looking for. The house of cards built up by high frequency trading, derivatives, futures and other hugely abstracted banking and trading concepts is far removed from this wealth creation.

Society rewards bankers with large amounts of money. Paul Graham argues that Steve Jobs increases material wealth, and so his monetary wealth in return was deserved. That's almost certainly true, but banking isn't a case of the average person going out and buying an iPad because it increases their material wealth. Bankers are rewarded richly because they've constructed a system over the last hundred years which takes large rewards for itself. There is very little true wealth produced by the system.

That essay was however written during the good times. In 2004 none of us were aware of just what a damaged financial system our economies are based on. In 2004 the families featured in that BBC documentary had a house with a hot tub. While I agree that there needs to be an incentive for productive working, there also needs to be a safety net. In our technology driven society, people shouldn't be struggling hard just to stay under a roof and feed their children. History has shown that the rich are the ones who have a voice, and so craft the system to benefit them.