> Doesn't this just mean that IQ Scores don't measure what they're supposed to?

The most cautious defensible position, in my opinion, is that "we can crudely measure a quantity that is related to the folk notion of intelligence"[1]. IQ tests are good, not great, at measuring something which 1) correlates to performance on just about any task that we regard as "mental", and 2) which we are thus far unable to divide any further – it appears to be impossible to split this "something" into a bundle of uncorrelated disparate abilities[2]. This "something" is called general intelligence, or g. However, all IQ tests are trainable and have different emphases; there is no bulletproof g test.

> My understanding is that IQ scores measure how smart you COULD be, all things considered.

The g factor is not a potential. It is defined by statistically extracting a factor common to all (or all minus epsilon) measures of intelligence, which factor is then found – empirically – to predict future performance. It does appear to measure a kind of intellectual potential[3], which is one reason psychologists are interested in it.

So, does this notion of general intelligence contradict the findings linked in this post? Logically, not at all; there is no reason a priori why the general factor should not be malleable. The question is if the general factor is in fact altered. Not an easy question to answer, due to the fact that g, being a statistical construct, cannot directly be measured, but must be inferred from other observations and tests.[4]

Since the linked Brinch-Galloway paper was published in 2011, others have attempted to determine whether education changes g[5]. Briefly, the authors measure a set of other cognitive skills, only partially correlated with g, but which cognitive tests also measure, and compare three models:

A) that education affects these skills via g;
B) that education partially affects these skills via g, but also directly affects some of these skills;
C) that education does not affect g, but only these domain-specific cognitive skills. In this last model, the positive effects of education on cognitive test scores are fully mediated by domain-specific improvements.

They find that Model C is favoured (p=0.04). The state of play is that education appears to improve performance in things like "Logical Memory" (the capacity to recall ideas and the logical connexions between them), and that this is what causes gains on IQ tests.

Practically, this means that education is good, very good even, but that there's something that it doesn't seem to touch. At this point one may wonder what exactly is so great about g compared to domain-specific cognitive skills. For probably the majority of jobs out there, domain-specific skills are exactly what are required; provided one has some moderate level of general intelligence to cover exceptional situations, the rest is trainable. If this becomes the received wisdom I wouldn't exactly be upset, as it would likely take a good deal of the heat out of "the IQ controversy". Perhaps instead of "general intelligence" we would refer to "abstract generalization capacity" or "cross-domain transfer ". With the great g threat on the back-burner for the time being, the discussion on trends towards increased demand for generalization skills on the decadal scale would then have room to breathe. Some of the proposed solutions[6] are rather radical for the current political climate...

[1] Steve Hsu, http://infoproc.blogspot.co.il/2013/04/myths-sisyphus-and-g....
[2] Not for want of trying – most prominently Howard Gardner's theory of multiple intelligences, which is quite popular outside of psychology, but which lacks empirical evidence (Waterhouse (2016), http://dx.doi.org/10.1207/s15326985ep4104_5)
[3] With a few exceptions, by far the most significance of which is childhood neglect, as commented by bildung at https://news.ycombinator.com/item?id=13183521 . But his point on motivation as a factor is misplaced – yes, motivation matters, but on average g will also have its effect. It is the combination of g and other specific factors which predicts any given facet of an individual's life.
[4] All this hairsplitting is an unfortunate consequence of the lack of a firm reductionist model of g. If we had such a model (grounded in neuroscience, for instance) we could just study the effect of educational interventions on the brain directly. Of course we can (and do) do this anyway, but it tells us a lot less without the model.
[5] Ritchie, Bates and Deary (2015), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4445388/
[6] http://www.businessinsider.com/how-to-genetically-engineer-i...