Well said. I find that most that are trying to become great programmers place more emphasis on finding a great team above a great project. Of course, having a great team may be one of your criteria for what makes a project great.

I don't think the path to greatness or the concept of programmer skill are as opaque as you suggest, even for non-technical evaluators of talent, once we at least have some small set of evidence and indicators available that tend to suggest talent.

For example, if I'm approached by an engineer from a major player like Google, I immediately have at least an idea that the engineer should be better than average, simply based on the fact that this engineer has already been vetted by Google's hiring process. That in itself isn't a guarantee of talent or 'fit' (fit isn't entirely relevant to this discussion), other than the ability to pass a rigorous interview which does not make one great, but it's an indicator, and chances are that engineer will probably be given a pass on a first phone screen. Someone who has held a job for years at an esteemed tech organization (need not be globally known) probably had to do something right to keep their job. Not foolproof, but fairly reliable.

Running references is another way to get some indicators. Granted, if a candidate can't find 3 people to say nice things about them he/she should find another line of work, but when you ask references the appropriate questions you will notice patterns if the references are being honest. I always ask about 'areas for improvement' as well as strengths, and if 3 references provide consistent answers to an area for improvement, I can be sure that is a flaw that should be acknowledged or addressed before hire. If you don't get consistent answers from the first few references, ask for more. Even a candidate's ability to provide references is another indicator of skill (not to mention soft skill, another element of greatness probably). Not foolproof, but usually reliable.

If working in smaller circles, you may know people that worked with this candidate that is not on their reference list. This is potentially dangerous territory to step into but I'd imagine a CEO making a large investment of both cash and equity would want to know what some others think.

Over the past few years, the technical interests outside of employment are becoming another indicator. I've said this before (as have others), but below-average engineers don't read or write Haskell code on weekends. This doesn't mean that all great engineers write Haskell on weekends, but that would certainly be an indicator of technical ability, and that indicator should have very few false positives.

The choices an engineer makes are also an indicator - not a guarantee, but certainly a signal. One might question the skills of someone who leaves a job doing very interesting and complex work for something much more junior or less challenging technically. These decisions start to create a narrative that at least suggest some skill level.

Of course, if we have the benefit of seeing someone's portfolio that is another indicator. It's difficult to judge an individual contribution to a group effort, but taking that type of evidence combined with any solo effort, interview performance, and perhaps a pairing exercise with your company's 'most talented' two developers, shoof provide some body of evidence for a decent talent evaluation.

A non-technical CEO could use similar tactics (not identical) to learn about a CTO, rather than relying on luck or other methods.

What makes anyone great and who decides who is great is tricky business, but it's mostly tricky when we have no prior info to use in the evaluation. The path is perhaps a bit more opaque, but I don't think most programmers really believe they are in the dark (and unable to evaluate talent accurately) as you suggest.

My article wasn't obviously as much about greatness as perceived value and salary disparity. None of the engineers in my article need to be great, or even good, for the disparity to exist. Your points, as always, are well made.