skysight-inc/HackerNews huggingface dataset:
Profile, Profile, Profile!

Use the 80-20 rule. 80% of your code's time is spent in 20% of the code.

Optimizing without good profiling is like painting a portrait blindfolded.

Pre-optimizing by coding your system to be "fast" before you get to profile it is like starting to paint a portrait before your model is even sitting.

When Kent Beck came in to optimize the Chrysler C3 payroll system, he discovered that one of the biggest time-sinks in the system was string concatenation!

It's also useless to optimize before you have a system that runs correctly. Kent also asked how he could validate the C3 system so he could optimize it. He was told it wasn't even producing correct results yet. He replied: "In that case I can make it run really fast!"
Very true, and I reckon the 'foundation to save the world' thing is a pretty commmon wish. (It's on my list, and on the list of a fair few people I know, but that could just be the circles I'm in). <naive> Perhaps we should all just club together (n% of profits from m successful startups) and give the Gates Foundation a run for its money... </naive>
Please quit posting blog-comments
I've noticed an on-again, off-again trend whereby people post their opinions on topics submitted to HN by posting an entirely new story with a link to their personal weblog. This is totally annoying, as we are unable to check out / vote on their ideas within the context of the original thread. Also, when you do this, cynics such as myself assume you are trying to take advantage of us for click-throughs, ad revenue, and/or improved search engine rankings. If pg were some evil content-hoarding troll, it would make sense to keep your comments/opinions within your own realm -- but he isn't, so you shouldn't.

I hope others here would agree that we should only encourage people to submit links from their weblog if they are articulating an entirely new, original thought, or a substantial enhancement to something that was previously discussed.
The argument against this I've heard is that if a dog could think 1000x faster, it would still only have dog thoughts but a 1000x more of them.

There must be something more to intelligence than just speed. Perhaps working memory is very important? Perhaps it's just having the right software in your head i.e., knowing how to solve problems and frame things?
New Jersey (though 26 miles from the city is close enough I should have just conceded to the NYC crowd) originally.

Currently - Philadelphia. Hence the nickname.
Using Rack Middleware to Handle Single Sign On
Ask HN: Do you trust Amazon S3 or Mosso Cloudfiles not to lose or corrupt your data?
Do you treat these systems as "reliable" and "safe" backends (since they're physically and geographically replicated) or do you feel it's still necessary to back up data hosted on their services?

How/where do you back up that data to, especially with large amounts of data (e.g. 100s of GB+)?
Because greed isn't necessarily good or bad.

It's just another variable amongst others with temporally varying degrees of importance.
Misunderstanding correlations among defaults was part of it but the other big surprise was that different asset classes which less independent than thought.

For example, no one expected an increase in mortage defaults to be accomplied by problems in stocks.

In fact, its likely that these two things fed each other, that the unexpected correlation in defaults is at least partly due to the unforseen correlation across asset classes and the unforseen correlation across asset classes is at least partly due to the unexpected correlation in defaults.
I'll plug myself here... :) If you are using Rails, take a look at http://railskits.com/saas/ -- a great starting point for your Rails apps to not only do the recurring billing (with Auth.net, Braintree, etc.) but also do account management, upgrades/downgrades, etc.

It has a lot of the work done for you, so you just plug it in and go.
GNU/Linux and Commercial Game Developers
Debouncing Javascript Methods
You could click "threads" (at the top left) instead.
I'd say that the main problem then isn't the income - it's just that the person receiving it can't manage his or her own money. It's also funny that the same person is considered the best person to manage a big corporation...

If people like that are the best available for the job, it's just scary.
should the guys who drive an armored car that carries millions of dollars in bonds get paid more than the guys that drive an armored car that only carries thousands of dollars in cash? Does the amount of money handled change the difficult of the work?

I think this is an interesting aside in the article. Seth seems to be arguing that it's the same work, but it really isn't. The amount of money handled does change the difficulty. First, it's going to require more trips to move the money and that is increased labor. More importantly, the risk goes up the more money you're transporting. Maybe those changes are small. OK.

The real difference comes in the value to the firm hiring you. If you're being entrusted with $1M vs $100K, the firm has a greater interest in making sure you're good at your job. Loosing $1M because you hired a poor driver is worse than loosing $100K because you hired a poor driver. Likewise, when we're talking about hiring for a CEO, a firm will want the best since they're entrusting that person with the livelihood of the firm. His or her job may be no more difficult than anyone else's job, but the firm has a high interest in making sure the CEO is competent. An incompetent person in an entry-level position is unlikely to be in a position to do the damage that an incompetent CEO could do to the firm.

So, by paying high salaries, firms hope to be able to take their pick of managers and get the most competent. However, firms have yet to show that they are able to determine the competent from the incompetent. So, at the moment, most firms are simply paying high salaries to whomever is good at politicking and showing themselves in a good light regardless of actual competence.

There is a theory behind it and it's not just marketing. However, the fact that firms have proven unable to filter the competent from the incompetent limits the applicability of that theory.
Reading Test Dummies
me three!
To first order, the number of instructions executed is not at the top of the list of things that make programs slow.

IO is. And, if you're not IO bound, you're probably memory bound.

Yes, caches help a lot, but a memory cache miss is 1-400 cycles and IO is far more expensive. In other words, cosine is faster than a cache miss.
I agree with you but I do believe the iPhone is a far more viable "$100" laptop — or at least has the potential to be.

I'm also happy to see consumers are starting to get irritated with the volume-driven top charts that were copy 'n pasted from the music store. $.99 Music Downloads != Applications. Hopefully Apple will listen and make adjustments soon, because for the past 6+ months they've ignored developer complaints in this regard. It's taken over 6 months to make very minor adjustments to the rating system.
I think you are focusing heavily on one particular model of an 'intelligence algorithm'. The model you are assuming sounds a lot like the type used for chess program. I played along with the 'culling algorithm' bit for the sake of convenience. But I'm not sure there is good enough evidence to assume that as the underlying model.

The definition of speed I am using is analogous to the clock speed or memory retrieval speed in a computer and it does seem at one level to be the way you are thinking of it as well. I think this fits in with what the researchers are measuring.

> The reason I didn't mention the possible variation in working memory capacity was a single word that I left out, leaving it, like I said, to the Computer Scientist: "exponential."

The memory requirements of a particular algorithm don't necessarily grow exponentially with the solution space or the input data. As with time complexity common space complexity relationships to the input size are n (you need to store at least the input), nlog(n), n^k and k^n. Most tractable problems would be nlog(n) or at worst n^k where k was small. The implication of this is that often small differences in memory can make a huge difference. Possibly you are forgetting that every extra bit of memory 'exponentially' increases the number of options it that can represent.

One of the first things you learn in CS algorithms is that the an efficient (let's say nlog(n)) algorithm running on a very slow computer will beat an inefficient one (say n^2) on a much faster computer. Sometimes resources such as memory will make the difference between being able to use an nlog(n) rather then a n^2 algorithm. To me this is a good analogue to apply to intelligence. For particular classes of problems (the harder ones) the better algorithm will trump a faster processor. Having a store of 'right algorithms' probably has a lot more to do with intelligence than raw thought speed.

Your view seems to be that the fundamental thinking algorithm is a fixed tree-pruning-like one. If it was then yes speed would be the main differentiating factor. I think this is the crux of our disagreement.
Just use both at the same time. Use S3 for active stuff, and Mosso as your secondary backup. The chances of S3 and Mosso crapping out at the same time are pretty much nill. And the cost of hosting something on S3/Mosso, as a one time backup is dirt cheap
Well, they bought JD because they wanted to market their cloud storage system. Its a powerful tool for gaining non developer users.
I think all of your points boil down to the same mantra as the article:

1. is the human variant.
3. finding which thing to "do less" of (benchmarking/profiling seem to be the "do less" ways of doing this task).
4 from the low level CPU perspective.

As I see it, all "four stages of optimization" are the application of the same rule just from different perspectives.
Both work beautifully! Thanks!
Armored car drivers do make more than most employees who do cash drops for small biz. "Armored cars" that schlep records are typically driven by folks who make less than those who transport money.

In short, there are at least some situations where the amount of cash handled affects compensation.
By more global I just meant that @foo would effectively put everyone under the same domain. Email just feels more fragmented.

Edit: also email puts restrictions on your identity. You're foo at some domain. @foo lets you pick a unique name that's valid online once you're old enough to pick who you want to be, without the footnote of _domain_ attached. Sort of how we all recognize Paul Graham as "pg", which fits his style.
I agree about the developers that Rails attracts. The one job I had with RoR, the employer was technically incompetent and he didn't pay. He was migrating _everything_ over to Ruby on Rails and spoke incessantly about it.

I've also noticed that the Mac crowd is extremely attracted to Rails. That should show us something -- and please don't assume that I mean all users of these techs fall in here, because I know that very intelligent people use both -- but Rails and Mac both seem to attract arrogant, know-nothing hipsters.

That said, frameworks are a pain when they're overused, but they're handy, even in advanced apps, when you simply consider them as a set of macros and convenience functions instead of an entire philosophy. This is something I really love about Pylons; it's there to help out when its needed and it stays out of the way the rest of the time.
Since "campaigning for president" is undefinable, trying to limit it is absurd.

In some real sense, every US Senator and Governor is campaigning for president.
Hulu Gains 10 Million Viewers In February, Now No. 4 Video Site In U.S.
Agreed. Seth Godin is assuming the only factor that should affect pay is the difficulty of the work.
I'm personally fascinated by the notion that efficiency and transformation seem to be intertwined. Hard problems can sometimes seem simple with a slight transformation in either the problem description or the data. Even the article is suggesting that to write fast code, transform your perspective of the problem. Do less. There seems to be something fundamental here.
Government Intervention in the Market for Toxic Cars
TwitterAuth: Plugin for Near-Instant Twitter Rails Apps
I have very little business experience, but I would guess this is less the case with engineers. Nerds care a little more about substance because that's what their job entails. Meanwhile, business guys care about how one sells oneself because that's their trade and they enjoy seeing it done well.
I set this up to backup my Macbook. I setup a RAID 1 mirror on two external USB drives on a Mac Mini, shared the drive, and I back up to it as a time capsule using this trick: http://www.readynas.com/?p=253 Then I ran a nightly backup of that volume to Jungledisk. It was very emotionally satisfying.

Unfortunately my up tube is too small, and the data won't fit. The initial 100GB upload was gonna take 40 days, so I stopped it after 10 with slow internets.

If you have the tubes, I do recommend it though. Its a really solid backup setup.
How VCs Determine Early Stage Valuations: Target Ownership
Tips for the Sophisticated Fugitive
I don't think YC really fits here, since you know ahead of time what their offer will be.
+1 Scotland (Glasgow area)
No. It's more like this: there's a massive prestige cost in paying lower than "market", and a benefit to paying more.

It's actually much cheaper to overpay executives and at-pay or slightly underpay junior people, which is what most "prestige" firms do.
You can easily get dual (possibly quad?)-cpu AMD mobo's and add to the power. It's the server line you'd use but same basic technology. $500 USD would get you a single Phenom 940 with 8GB of ECC ram, computers are cheap these days.
Could you explain that a little more? If it only needs 1MB of memory I would assume it fit's on most modern CPU's L2 cache. At which point I don't see why it would get slower if you added L3 cache etc.
Cambridge, UK
(Note: I'm the founder of a cloud-based load testing provider...)

Well, there are a lot of reasons why cloud-based load testing makes sense beyond scripting. But specifically about scripting, I believe that the cloud actually _does_ make script creation MUCH easier and script playback MUCH more realistic. Here's why:

Traditional load testing tools emulate HTTP traffic. Their recorders watch the protocol-level traffic a browser makes, and then poor engineers have to disassemble the traffic to make it appear as if thousands of users are coming from one machine.

They end up writing regex expressions and other such logic which effectively emulates the more expensive parsing a browser does. Often times, especially with rich AJAX apps, they end up duplicate much of the same code written as JavaScript to run in the browser, but this time inside of their load tool. For an example, see my article on why load testing is hard when it comes to AJAX:

http://ajaxian.com/archives/why-load-testing-ajax-is-hard

With cloud-based scalability, a new opportunity is here: to ditch the old ways of load testing and actually use real browsers. Scripts become way simpler because you're no longer scripting complex protocol-level traffic, but instead scripting simpler user behavior. And the script playback becomes more authenticate because you are guaranteed not to have a bug in the protocol-level playback, error reporting can show you what the user actually saw with screenshots, and the traffic pattern is realistic (two connections per host, etc).

Of course, not many providers are doing this (yet). My company, BrowserMob, is. Thanks to the cloud, instead of allocating 2GB and 2 CPU cores to a load test, we might allocate 2TB and 2000 CPU cores. But because it's elastic usage, our cost is still low, so our prices stay low too. All of the scripting is done with Selenium, an open source browser automation technology I am a contributor for.

If you're interested in trying it out, we provide $100 worth of free trial credits:

http://browsermob.com

Besides the browser vs virtual user stuff, there are definitely plenty of other reasons why the cloud is a great way to do load testing, but I think those have already been touched in the comments.
Sure? It's like anything else, don't trust a single solution. In addition to local backups S3 is good, in addition to S3, Mosso. There will always be accidents, data loss, corruption, etc. The only way to mitigate that risk is simply to cover your bases and avoid relying on a single thing.
Exactly!

So rare that this debate actually gets to the real point, which is that the BSD license and the GPL optimized for two different "freedoms". Ne'er the twain shall meet.

Which is larger, a company of 100 selling $500 million a year of goods, or a company of 1000 selling $50 million a year of goods? Well, it depends on your metric. Two perfectly valid metrics produce two different results.
Interesting post... I have submitted my response here: http://tinyurl.com/cdb8qx
Law firms tend to be partnerships, so if partners want to give a new guy a ton of their own cash, good for them.

Wall street transformed itself from mostly partnerships, where partners invested their own money along with their customer's money, to corporations where professionals invest other people's money.

Apart from that, we have the system of deciding compensation for CEOs. The CEO picks people to sit on a compensation board, and they then recommend/decide what the CEO should get paid.

Warren Buffet in all his years of being an investor has been asked to sit on a compensation committee exactly once.

In conclusion, it's the system stupid.

Obviously CEO's salaries are going to go as high as can be in the current system.

Obviously people investing OTHER people's money are going to take more risk then people investing their own money.
This sounds very much like something the SA Air Force did with their mirages in the Bush war of the 70's and 80's.

BTW - There is a book written about the history of the Mirage in the SAAF called 'vlamgat' (lit. fire/fiery - behind/ass). Try http://www.saairforce.co.za/reviews/33/vlamgat-cd.

Anyway. In this book, which is written by an ex-pilot, a maneuver is described which sounds very similar to the pop-up. The aircraft also approaches the target at low altitude, then pitches up. As the pilots pull up on the stick they release their bombs, but continue climbing up and away. The bombs (old fashioned, dumb bombs) would then fly in a parabolic trajectory like you'd expect and land somewhere near the target ;)

If I remember correctly this tactic was actually used in combat with surprisingly effective results. I say surprising because nobody believed you could actually hit a target with a bomb that flies in a trajectory with a horizontal length of a few 100 to 1000 meters and was released on the pilot's 'gut' feeling...
SpiralFrog Goes Under
every US Senator and Governor is campaigning for president

Those who are not "natural born citizens" of the United States (such as the current governor of California) cannot become President, by the Constitution, but, yes, for the most part most politicians of national prominence keep a future run for the presidency in mind in most of what they do.
I agree with you 100% and use @dawson on almost all communications, the fact it's now recognised as a Twitter handle by the majority just shows how far the service has come.
Should "Your site has queued." be "Your site has been queued."?
Got Clawbacks? Thugz on the Bailout - Freakonomics Blog
You know what market capitalization is?
I've never seen anyone do this...got an example?
I wish you'd use LangPop.com for stats instead of TIOBE:-) I agree about the marketing/communication aspect of things though: I was under some sort of impression that 1.9 was, like "odd numbered Linux kernels" some sort of permanent-beta. I'm not quite sure where I got that idea, but a few other people I know seemed to be under the same impression. Something else that might help is a prominently located guide/tutorial to upgrading to 1.9.
Just because 1000 people want the job and are "qualified" doesn't mean they would do as good a job. Consider this example:

Company A has 1000 employees that they pay 100 dollars a day, totaling $100,000.
They can hire Joe to be COO for $1000 a day or Zach for $5000 a day. Based on both mens track record the company estimates that Joe will increase efficiency 1% while Zach will increase efficiency 10%. Since a 10% increase in efficiency will likely gross them $10,000 a week and only cost them $5000 (Zach's salary) they hire him instead of Joe.

The same thing happens all over the place. Someone that can make a 10% change company wide is worth FAR more than someone that can only make a 1% change.
Emailing the password? Ha.
Haha I was going to make the same joke damn you :)

(for the downvoters who never clicked through: it links back here ;))

edit: not sure people appreciate irony :P
> Supply-side policies are necessary here in order to bring down materials costs and to continue to build more housing to bring prices down.

In a large number of places, housing prices are not all that dependent on materials costs. (Or rather, other things drive up the price to the point that people spend more on materials because that doesn't have much effect on the overall price.) Instead, they're driven by land use and other policies.

> Housing is not an investment, it should never be treated as an investment.

Housing is a large pile of resources. While it's a durable good for consumers, it's an investment for housing providers, builders, and so on.

More to the point, things that people do for profit are far more likely to happen than things that people do "for the common good" or some other reason. And the desire for profit is what drives prices down.

Investment is a great way to get profit involved.
Atwood just ensured this article will show up high in any Google search for expert's exchange.

As much as I'm not sure if his writing is really helpful to good software engineers, I applaud him and Joel for taking on expert's exchange.
I've got little love for Mosso and their offerings. Rackspace as a whole has never done me wrong but I have wasted many hours working with their techs dealing with some of their server problems on Mosso (this is for hosting not cloudfiles). I eventually had to ask for my money back which they gave me but not before they tried everything in their power to keep me working with them. (Which by that point I considered a waste of time and migrated clients over to SliceHost which later got bought out by Rackspace...)

So while I may be biased - I'd like to point out that a bug in a end user product brought down their entire cloudfiles platform and that the same bug also effected S3 and had no noticeable effect.
If it moves, tax it. If it keeps moving, REGULATE it. And if it stops moving, subsidize it. --Reagon
I'm trying to think where this first appeared, was it Salon?
5 - 10 years ago?

-- EDIT --

It was Salon: http://archive.salon.com/21st/chal/1998/02/10chal2.html

Feb. 10, 1998, jeez I'm old.
Hm, Rails still doesn't work fully on 1.9.1. There's a few bugs relating to character encoding in templates still, you can find details on Lighthouse. It mostly works though. I've switched exclusively to 1.9.1 for console work, which is unaffected by the ActionView bug.

I agree with the point about how projects not being ported to 1.9 begin to look unmaintained. It's been 3 months in stable now, and a year of betas before that - I can't help but start thinking any project not moving as fast as possible towards compatibility might not be one anyone should be relying on.

Most unforgivable offenders: Mongrel and the DB libraries

Most pleasant surprise: RMagick!