
Fizzbuzzed - Alupis
http://dominickm.com/?p=514
======
tptacek
"[W]hat is your favorite programming language / platform and why" is less a
predictive interview question than it is a Rorschach blot _for the
interviewer_. 

In most established tech companies --- those with, say, more than 10 employees
--- interviewing is performed by a semi-arbitrary selection of line
developers. 

If you're managing the hiring process at one of those companies, you should
keep in mind that for a typical software developer, being involved in the
hiring process is the closest they're going to get to participating in
corporate strategy.  For the time that they're in the room with a candidate,
they are essentially the CEO of the hiring process.  The more attention I've
paid to hiring processes, the more convinced I am that this goes to people's
heads.  In no other job capacity do software developer's subjective beliefs and
opinions get applied so directly to business outcomes. 

~~~
justinpaulson
I completely disagree that the question is a bad question.  He does not say
there is a right answer, only that they are able to make a competent argument
about what language they like and why.  The point of the question is to see if
they have actually put much thought into developing software.  Enough thought,
in fact, to have an opinion on what they like and why.  Furthermore, it tests
their ability to verbalize convincing arguments for their opinions with
logical evidence for those arguments.  These are skills that are needed when
working in a development team. 

Now if he is saying, "well they said python and I hate python, so they are
wrong!"  Then, obviously, that is not a good interview tactic.  But I didn't
find that to be the case from the way it was presented in the article. 

~~~
tptacek
The problem isn't that there's a wrong answer, it's that it doesn't
immediately admit apples-apples comparisons between candidates, it's hard to
structure it so it can do that, and even harder (probably requiring several
generations of candidates) to tune the apples-apples criteria to find
something that actually predicts how candidates will actually perform the job. 

Those are the big theoretical problems with questions like that.  But the
pragmatic problems with it are even worse: they are a lens that focuses and
applies subjective biases on the part of interviewers.  Developers are terrible
psychologists, and subjective interviews are the one time that we demand that
software developers perform as psychologists. 

The best job interview questions are strictly and (as closely as possible)
objectively predictive, and the easiest predictive questions to ask are work-
sample questions: "here is a problem we solve routinely in this job,
candidate, please solve it yourself." 

~~~
justinpaulson
I still completely disagree with you.  You are misunderstanding the point of
the question, and probably missing the importance of being able to formulate
an opinionated argument about development and present it to a team.  That is
the point of the question. 

You should not be apples-apples comparing candidates anyway.  Everyone is
different, and it is a collection of skills that you need to be a developer. 

Candidate A may be a person who isn't as quick with coding, but they can
quickly architect a solution in their mind and formulate an opinionated
argument about why it is superior. 

Candidate B may be an incredible coder, but unable to explain their
methodology, or architect a solution that is generic and re-usable. 

Both of these candidates can be an asset to the team in different ways.  I find
a lot of developers, the "JUST LET ME CODE" types, really don't understand the
communication skills needed to develop an application with a large group of
people.  This question helps find people that will be better in that group
situation, regardless of how strong their coding skills may be...because
coding is only a portion of the job, in the real world. 

Edit: I also pointed out in my reply, that an interviewer who takes a
subjective bias when assessing the answer to this question is doing it wrong. 

~~~
tptacek
You believe that you shouldn't try to hire based on apples-to-apples
comparisons of developers.  I not only don't agree with you, but I run a
(relative to startups) high-volume recruiting practice premised on that
belief.  We're not going to agree, and that's fine. 

In the 12 years of my career prior to discovering objective, work-sample,
standardized, data-driven hiring, I hired exactly the way you're advocating.  I
sought out questions like the one you're advocating for.  My favorite question
from that time: "if you had to put together a libyou.a that you take from
project to project, what would be in that library?"  I found it to be a great
way to start a conversation with a candidate to get inside of their head, and
also liked the fact that it quickly gave me a signal as to whether the
candidate had actually done the kind of work we did in our shop. 

As it turns out, the ability to project a signal as to whether a candidate has
actually done the kind of work we do in our firm is _not actually useful_. 
What's useful is: "if the candidate is put in a reasonable position to try,
can they actually perform the tasks the role requires".  Surprisingly --- in
fact, shockingly --- I've found _zero correlation_ between the capacity to
_signal ability_ and the capacity to _demonstrate ability_.  ZERO.  Our best
performers six months in, as often as not, interview terribly on "signalling"
terms, and were hired because work-sample testing overrode those concerns. 

------
jpetersonmn
I've been programming for a couple years now, part time.  I work in IT
troubleshooting proprietary software on customer's machines.  I've written
several vb.net applications to automate things in the office. ( Screen
scraping/website automation, log file readers/parsers,, etc...) Lately I've
been working in python more for my screen scraping needs and also learning
web2py.  I've thought about trying to apply for a programming job where I work,
or at least maybe see if I can shadow a programmer to see what it's like. 

I've always been put off by thinking that I'd just be embarrassed because I'm
not good enough and they'd laugh at my code.  Nobody in my group at work does
any kind of coding or scripting, and all my work is proprietary and I can't
really share it on the internet for feedback.  I've heard of this fizzbuzz test
before, but just now looked up what it is exactly.  I can't believe how easy
that is, and that people are actually failing at it.  Makes me feel like maybe
I should apply for some of the developer jobs at our company. 

~~~
bichiliad
To clarify, I don't think "Fizzbuzz" necessarily means "printing out Fizz on
multiples of 3, Buzz on multiples of 5, etc."  Fizzbuzz generally refers to a
simple programming test to make sure candidates can actually write some amount
of code. 

~~~
jeffasinger
I've interviewed people for a junior position recently.  I gave the FizzBuzz
test, verbatim from Wikipedia.  Only about 10% were able to do it on a
whiteboard, without help. 

~~~
opendais
I can't write any code effectively on a whiteboard in front of an audience. - >
I'm a terrible programmer. 

TIL I'm a terrible programmer. 

~~~
pkaye
An audience of one.  What about on a piece of paper?  In front of a computer. 
How would do in code reviews?  You can state your concerns to the interviewer
to see if they can accommodate. 

~~~
opendais
I've never seen a whiteboard interview with only one interviewer.  The smallest
was 2. 

Whiteboards are for diagramming things more complicated than lines of code. 
[e.g. Interactions between several web services or paths of control across
entire programs]

I've honestly just accepted that by the standards of many people's
interviewing processes I'm a bad programmer. :) 

------
xrange
>Pretty much everyone I asked to do Fizzbuzz failed

…I can't help but wonder what that really means?  They gave up without
producing any work?  The program didn’t compile because they were typing random
characters into the IDE?  They were unfamiliar with the IDE, and couldn’t find
the "compile" button?  They couldn’t get the program to compile because of
missing semicolons?  The program didn’t write anything out to the terminal?  The
program never printed out "Fizz"?  They printed out "FizzBuzz" on every line? 
The program never printed out numbers?  The program printed out commas instead
of newlines after every number?  They went from 1 to 101 instead of 1 to 100? 

~~~
jasode
>I can't help but wonder what that really means? 

My guess: it's the unfamiliarity of the mathematical modulus operator that
trips most people up.  A lot of CRUD developers will never need to use modulus
calculations in day-to-day work.  Pull some values from a database record,
throw it up on a form, then save it back.  Use plus+ for totaling dollar
amounts.  Use multiply* for calculating sales tax.  A LOB programmer can do
stuff like that for years without modulus% anywhere in his code.  Something
like credit-card number validation does use modulus calc but programmers can
just copypaste[1] someone else's example or just use a library. 

However, a lot of these FizzBuzz discussions talk about _computer science_
graduates specifically.  A lot of CRUD devs do not have formal comp sci
backgrounds so it's somewhat understandable that they're not familiar with
modulus.  However, I can't think of a reason why a compsci grad wouldn't know
it. 

[1][http://www.codeproject.com/Tips/515367/Validate-credit-
card-...](http://www.codeproject.com/Tips/515367/Validate-credit-card-number-
with-Mod-algorithm)

~~~
xrange
>My guess: it's the unfamiliarity of the mathematical modulus operator that
trips most people up. 

This is an interesting hypothesis, can anyone confirm that this is a
particular sore spot?  I wonder if that would show up as a generational gap?  If
the younger generations are working with web pages and never used much
arithmetic, let alone remainders.  But older generations grew up with low level
graphics programming and arrays in languages like C and assembly where the
modulus comes up all the time.  So could we come up with the moral equivalent
of FizzBuzz that a much larger percentage of the people who failed FizzBuzz
could pass?  And then I'd like to see how hard I'd think that test was. 

------
jasode
>There is however one last hurdle — a written test in the form of Fizzbuzz. 

A "last hurdle"?  Shouldn't it be the first hurdle? 

I thought the point of using FizzBuzz in its proper context was to be a quick
filter to weed out the majority of candidates.  That way, you avoid wasting
interview time with them. 

~~~
eli
I don't like making job applicants do work -- even something as simple as
fizzbuzz -- if it's unlikely I'm going to hire them.  Just seems unfair.  Also,
if I'm giving someone a test, I'd prefer they work on it in front of me, which
means it won't be the first screen.  And finally, I don't think struggling on
FizzBuzz _necessarily_ means "No Hire." 

(I actually don't really give these sort of tests at all anymore.  Didn't seem
that useful in practice.) 

~~~
jasode
> \-- if it's unlikely I'm going to hire them. 

But how do you _get_ to the "unlikely" determination?  Do you want to get a
ballpark reading on "unlikely" within the first 10 minutes of the interview or
at the very end _after_ investing an hour? 

I'm not suggesting that FizzBuzz is the be-all-end-all but for the folks who
_already do_ find useful as a low-investment bs filter, it makes the most
sense to use it at the beginning of the process and not at the end.  Using it
as a " _last hurdle_ " makes no sense if you have a stack of resumes to plow
through.  If you have prior assumptions that 90% of those candidates are not
competent at programming, what's the logic in using FizzBuzz as the last step? 
You want to identify the 10% as quickly as possible before investing too much
extended interview time. 

~~~
eli
Yup, you're probably right.  If you're at the end of the process and on the
cusp of hiring someone to write code but you're not sure whether they can
write simple code then there's probably something wrong with your process. 

------
logfromblammo
FizzBuzz is just a bozo filter.  Sure, it screens out unqualified applicants. 
But why are you even getting unqualified applicants in the first place?  It's
because your hiring strategy is fundamentally flawed. 

When I go out to look at job postings, that is all that I see.  Single,
standalone jobs.  What I really need to see is an entire career track within
the same company, from green rookie all the way up to the highest non-
executive position on that track. 

If I saw that, I could self-select for the position that is most appropriate
to my current skill level.  You wouldn't need to use FizzBuzz as my bozo
filter.  You would have to use a more difficult task, such as prioritizing a
list of bugs and feature requests in a hypothetical program, based on
perceived severity, resource requirements, and business needs. 

But it seems like companies now only ever want developers with 5-10 years of
experience in all the techs from the stack that they already use, and there
are no promotions except into management.  Where do they imagine those people
come from?  Where do they think those people go after they age out? 

Is there some developer nursery out there that takes green rookies, trains
them up into competent developers, and then just forces them to leave?  Is
there a developer nursing home that hires only grizzled, jaded old veterans
and pays them all handsomely to do work that never involves any training? 

The reason that you need an ever-increasing amount of bozo filters is because
your job openings do not match the profile of job-seekers.  Unqualified
applicants are clogging your system because your posting is the closest thing
they can find to their own skills. 

Bozo filters treat the symptom.  The only cure is to give all those bozos their
own jobs to apply for.  If your company is large enough to support that many
positions, you really should establish complete career tracks, across all
levels of experience, and advertise them as a whole to job seekers, so they
can tell you what rung of the ladder they think they should be on. 

In short, you advertise all the positions that already exist at your company,
the number of vacancies, and how those positions are linked by promotion. 
Anyone applying for multiple positions spanning more than two mutually
exclusive skill levels is automatically filtered out as a resume spammer. 

------
coreymgilmore
I've always been a fan of "walk the walk, then talk".  Basically, do FizzBuzz
first then follow up with questions.  Reason being, if you can't do some basic
coding than its not even worth the time of having a discussion.  You are, after
all, hiring for a coding based position. 

~~~
Scuttles
Doesn't seem very scalable if you have a lot of applicants (unless you let
them do the test at home?). 

I think asking for code samples up front is the best way to weed out those who
aren't good candidates -- you can tell a lot about someone through their work. 

~~~
dllthomas
A few places I've applied have given me coding challenges to work on from
home.  None has taken me more than an hour, were relatively simple and fun, and
I've never failed to get an interview afterward.  I'm not entirely sure what
all of that _means_. 

------
dcminter
I think this Spolsky article provides some insight into why this might be the
case:
[http://www.joelonsoftware.com/items/2005/01/27.html](http://www.joelonsoftware.com/items/2005/01/27.html)

~~~
lotsofcows
We request solutions to slightly harder problems in response to an initial
application.  Even though this is done at home, it still filters out a large
percentage of applicants. 

We then do something at the same level at the start of the interview.  This
filters out about half of the people who have got as far as an interview. 

We then do an SQL test, building slowly from "select * from user" to a multi-
table with sorting query.  This filters out the other half. 

We don't hire often. 

Anyway, this is a very long-winded way of agreeing with Joel.  I meet good
programmers all the time - just rarely in interviews. 

Alternatively, offering the going rate is too low an offer...

------
3pt14159
I struggled with this when I was responsible for hiring at 500px before I
moved on.  The key is to ask for code samples ahead of time.  It doesn't
necessarily have to be open source contributions, though that slightly
preferred, but I refused to interview someone that didn't have code samples. 

I went from 30 interviews per one hire to about 2 or 3.  The most common
problem being a potential candidate not knowing enough of the internet stack
to be truly useful (for example, knowing _only_ rails, but having no real
knowledge of how AJAX worked, or load balancers, or databases, or DNS, etc). 

~~~
jtratner2
How did you handle people with no open source contributions?  I'd imagine most
non-OSS code would be under an NDA or something. 

~~~
3pt14159
They just sent me code. 

It is up to them to make sure that they are following NDAs.  Also, sometimes
code isn't open source, and it isn't NDA; like 4th year assignments or small
contracts. 

------
izzydata
Makes me feel slightly more confident in my ability as someone who recently
finished my CS degree, but I still feel very far behind.  I find it very weird
that someone even after their freshman year in such a degree program could not
understand or comprehend such a simple task.  I don't think anyone in my
schools department would fail at these tasks after taking the introduction to
programming course. 

Where are all these students coming from?  Perhaps there is some problem with
the source.  It seems like they aren't being taught to understand, but are
instead being taught to copy instructions in some manner. 

------
lazyant
"[W]hat is your favorite programming language / platform and why" while a
reasonably good open-ended question, students just haven't been exposed for
long enough to different languages/platforms to have an opinion, although this
question (as almost any other) will certainly have the "hacker in my own time"
types talking. 

~~~
patrickmay
That's why I like it.  I add "and what would you change about it if you could?" 
to identify those candidates who really think about their tools. 

------
geetee
FizzBuzz is the programming interview version of asking UK Chancellor George
Osborne what 7x8 is[1]. 

[1][http://www.bbc.com/news/blogs-magazine-
monitor-28143553](http://www.bbc.com/news/blogs-magazine-monitor-28143553)

