I think that one of the huge problems is that there has never really been to my knowledge a systematic look at the epistemological limits of IQ tests. However just looking at it quickly I can spot two huge epistemological issues which make IQ tests more or less worthless for objective study, and more or less cultural artifacts.

The first, which I have mentioned in a few posts is that at some point you end up comparing apples and melons (the pun for those of you who didn't get it is that 'melon' is Classical Greek for 'apple'). These things may be things where we can make connections but any cultural weight we place on one relative to another is just that, a cultural weight. For this reason intelligence, even if IQ tests measure what we expect them to (and I am not sure of that, see below), the results cannot be objectively reduced to a single number. You would really need to look at intelligence as a multi-dimensional vector with the understanding that we might be missing some aspects anyway.

The second problem though that I can spot immediately is that some skills they want to test cannot be objectively tested. The big one that comes to mind is pattern recognition. If you haven't read it, I think Heisenberg's "Physics and Philosophy" is a must read here because his discussions of the relationship between data and theory are directly relevant to this.

When we recognize a pattern what we actually do is we project assumptions onto the data and find a pattern which fits both the assumptions and the data. This is the process Heisenberg articulates regarding the formation of scientific theories. The pattern recognized is both a product of our assumptions and the data and therefore patterns are at once recognized and created. Of course Heisenberg was talking about his forte (theoretical physics) but the same applies here because this is a human process.

The simple fact is that several competing sets of patterns can be made to fit the data. For example, take the following set of items:

Hammer, log, ax, saw

Which one doesn't belong?

One pattern might be to that "hammer, ax, and saw" are the pattern and log is the outsider. Another pattern might be "I can build something with the hammer, the log, and the saw, much more than I can with any other combination so the ax doesn't belong." Therefore such a question has at least two legitimate answers because two different set of assumptions can be projected onto the set with very different results. Now typically we are going to say one of those (the first, "log" doesn't fit) is correct and the other is wrong, but that doesn't actually test the ability to recognize patterns so much as it tests the ability of the test taker to figure out the answer that the author wants. This makes pattern recognition fundamentally culturally determined and if you can't get away from this you can't quantify this through standardized testing.

But let's take this further. We typically refer to a genius as someone who finds useful patterns that nobody else does. The IQ test then, if we accept Heisenberg's epistemology, should actively select against genius in these areas-- either the patterns must be trivial, or they must be sufficiently complex you cannot be sure whether a "wrong" answer in fact does not have a recognizable pattern to it. So you end up having to throw out a good chunk of the test here as being very closely culturally determined (and basically an exercise of mind-reading the author).

The first problem could be solved by expressing intelligence as a multidimensional vector, but the second is much more problematic. How do you know that the test answer is right and the test taker is wrong when it comes to two competing patterns seen in the question?