I have serious misgivings about tech interviews, though I do understand why people do them. While I don't think it's unreasonable to ask programmers to demonstrate strong programming skills (even on what seem to be academic-ish problems), the process is bad enough that it is a bit of a scourge on the industry, and I do think it deters talented people from entering the industry, remaining in the industry, or applying for new jobs or pursuing new opportunities that would be a good match.

I think we need to start calling them tech exams rather than tech interviews. I think many of the problems come from the fact that we're acting like an exam is an interview. If you think of it as an exam, then you realize most "exams" in your life comes with some rights and expectations on the part of the examinee, and some obligations on the part of the examiner.

Here's my ideal wish list:

1) An exam should come with an associated study path. A good exam doesn't give you a set of questions with solutions to memorize, but you should know roughly what will be tested and how to prepare for it.

2) An exam should be conducted by someone skilled enough to recognize novel but good solutions.

3) An exam should be graded in a consistent and fair manner.

4) An exam should lead to a publicly acknowledged and lasting credential (you passed the bar, or an actuarial exam, you got credit for a class, you obtained a degree).

5) Ideally, an exam will come with some sort of feedback if you fail, with some opportunity to retake (not an unlimited opportunity but some kind of structured opportunity).

I've been through a number of tech exams, and most of them would be 0/5 on this list. Even the best tech "interviews" provided extremely limited feedback on how I did, even the best gave only very general information about what would be tested. My university exams typically met all the above criteria, including my exit exam for an MS degree. Certification boards often do as well. Standardized testing exams generally let you know how you did (how many questions you missed, sometimes what questions you missed), and perhaps what the breakdown was.

But in tech? Sometimes you get a study path (Steve Yegge's advice about getting the job at google is very good, if high level). Often, you don't know what you'll be tested on. If you do fairly well, just not quite well enough to get a job offer, you may very well remain unaware of this. I understand this is because of liability, but that doesn't really make it better for a student who may have put fifty or more hours into studying and is back to square one.

Think of it this way - imagine if you took the bar exam and did quite well, but a top law firm decided not to hire you. As a result, you get no credit for how well you did do, nor do you really have any idea if you did miserably, poorly, fairly, well, or exceptionally well on the test. If you did well enough to pass the bar but not well enough to get hired, you'd still have a credential. Not so with tech tests.

I understand some people may say this is the market at work, the price we pay for not having a formal system of credentials. Honestly, I'd happily take a "bar exam" for software development if it meant I wouldn't have to do it over and over, each time I interview (though I am not in favor of a required exam to work as a software developer).

In general, I support tech companies right to do what they think is best. I am free to opt out if I feel it is unfair. If they want to quiz people on difficult data structures and algorithms problems, at a whiteboard rather than terminal with compiler, fail them if they don't make enough progress in 45 minutes, and accept a high "false negative" rate, that is their right. It only starts to rankle/infuriate me when these same companies start raising an alarm about the "shortage" of qualified software developers.

Just a final word here - I want to be sure people don't think I believe I'm entitled to this sort of thing from tech companies, that they owe me this kind of exam. I came up with this list as part of trying to get my thoughts straight on why (some) people hate tech interviews enough to give up on new jobs or the field. I believe it is because exams are extremely stressful, and that many of the institutions that make heavy use of exams (actuarial profession, the bar, universities) recognize this, and have adopted a set of obligations on their part - a kind of "if we're going to put you through the gauntlet, here's what we owe you in return". The second half of this informal but actually somewhat venerable relationship between academic institutions and students is missing from tech interviews.