This is the part I fundamentally disagree with and the reason that I am not a proponent of testing. Because it is loaded with so many assumptions on the part of the interviewer. I once failed to get a JavaScript gig because I did not use the arguments array in a method. The interviewer though that the question given a arbitrary amount of parameters passed into a JavaScript function how would you sum all of the parameters. I content that passing an infinite/arbitrary amount of parameters into a JavaScript function is something that I will never do. Not only that I would have a talk to any developer on my team that did, if it was not some very special need for such magic code. The interviewer thought that this was a perfectly reasonable filter and said so, it was stated as the reason that I did not get the position.

You see that's the crux of the issue, I can be a JavaScript expert and deliver massive JavaScript applications without having to resort to the arguments array to figure out how many parameters where passed into a function. Also, if I ever had a use case for it, I would look up the API and probably mentally discard it shortly afterwards. The interviewer thought that it was a perfectly reasonable filter, but yet I am one of the top JavaScript talents in my market. It did not work for them, because they assumed that knowledge of the arguments array was basic knowledge and any developer worth their salt would know it. Yet in my time working with some great JavaScript developers not a one of us ever used it. Further it harmed them because I now believe they are as incompetent as they believe me to be, as such if someone in my peer group asked about a position with them, I would relay my story. Not to be indignant but to be truthful, as such they lost access to valuable resources in a constrained market.

That being said, one can be an expert in say Java and not know the JDBC API, maybe all of their work revolved around JPA. Many in Java would argue that JDBC is fundamental and should be known to be an expert but a new crop of developers have never know anything but ORM. But we older developers assume that JDBC is a fundamental prerequisite because it was in our time. They cant see anything but the progression from JDBC to JPA because it is the life experience that they had. As such it's flawed and irrelevant. You have to be very careful to not bring interviewer bias into an interview and test are loaded with them. Even the simplest of test.

Does it not strike you that quite a few HN'ers openly admit that they would not pass the filter? Some of them long time contributors and members, some of them very respected? To me, I would question the viability of my assumptions if HN'ers where openly saying no your filter would eliminate me. I mean we have some of the best of the best lurking here. 37 has been complaining about it for a long time, I have had my own threads about it, and everyone agrees that hiring is flawed. Maybe it's time we reevaluate our assumptions based on what guys like the crew at 37 are doing since they seem to be improving on candidate selection over the rest of us.

The point I am making is what you are trying to garner, can be identified by asking a candidate to show you something they built and walk you through the routine they are most proud of. There are going to be functions, loops, if statement etc. in that code, if they can't explain what they built, then you should be suspect. Giving them a test just introduces bias of what the interviewer thinks they should know, and does nothing to tell the interviewer what they know.