It's actually kind of an interesting thought to consider. If programming challenges aren't good at helping you become a better programmer are they good for determining if you're a good programmer? These kinds of sites pop up because of how the engineering interview is conducted. I presume that the more engineers study to become better at these challenges the less effective these challenges will be at finding good engineers.

For example, maybe you get someone who can find the right data structure or algorithm to solve puzzles, but will that same person understand physical limitations of computer hardware and networks? Will they know how operating systems and CPUs work? Stuff like CPU interrupts, processes, memory management are important. The crafting of software is important, being able to design maintainable architecture, understand important design patterns, cache invalidation, testing, different programming paradigms.

At the other end of the spectrum of what I think programming interviews do wrong is test reference knowledge. Stuff you can look up online by Googling. Maybe you can gauge some amount of experience by familiarity of having been exposed to enough of some type of thing to have ended up memorizing a lot of documentation, but I would say not knowing documentation that is easily searchable online is not a good indicator of a bad candidate.

In my experience bad hires have tended to turn out to be bad for reasons orthogonal to programming challenges and reference knowledge:

#1 people with a bad attitude.

This can take several forms: mean to others is the first one. But bad attitude can also mean someone who isn't mean, but someone with a bad work ethic, calls in sick all the time, asks too much of others. Someone who is lazy or unwilling to learn from past mistakes and repeatedly breaks production code. Someone who doesn't actually like to program but does it for the paycheck. Someone who goes off and codes things by themselves without input from others and creates a lot of work for others. Someone who is politically motivated in the workplace and strives to make themselves look better at the expense of others. Someone who is insensitive to women. Someone who is insensitive to diversity. Problems with integrity.

Only twice have I come across an individual who was a good person with a good attitude, worked hard, but wrote shit code. Both of these people had a masters degree in computer science and were excellent at interview challenges. They didn't understand the code base, they wrote things that just didn't work. They wrote just really sloppy, bad logic that you didn't have to look through much to realize it wouldn't work. One guy ended up on a performance improvement plan and then quit under pressure, and the other sort of just fell in as a probably permanent junior programmer who needed supervision, but with help ended up adding value. His life though is one of very basic implementation of other people's designs. I don't know if he ever got better.

My point is, the modern programming interview fails in many ways. Still even I use it. There just isn't enough time in an interview to figure out if a person will be good and enough. I also think 3 or 4 rounds of phone interviews and in person is too stressful and time consuming and anxiety causing for a candidate, and not worth it.

The best hires have always been in network hires. Hiring someone based on someone else you knew. Those kinds of hires didn't even have formal interviews sometimes, you just knew they were good based on reputation. This also has problems, because maybe you just aren't in a network so you can't get hired this way, or maybe you've exhausted the network available to you and your employees when trying to hire.

Anyway, these programming sites are pretty fun. So I like it.