skysight-inc/HackerNews huggingface dataset:
"I don't think the SAT thing is very interesting: presumably (or possibly, at least) it is still a culture thing that women are less interested in maths, which would explain the gap by the end of high school."

That could be it. But I'd actually be more worried about students who were just "less interested" than students who were interested and tried really hard.

"The IQ test people also claim their tests are culture insensitive, but that is definitely not true. Just think about the following kind of test: show 10 kanji symbols on a page of the book and on the next page ask the reader to identify them."

Actually, what IQ testers found was that the culturally insensitive tests exacerbate group differences. Unless there's something cultural in being able to remember a sequence of numbers and recite it in reverse, or to have high reaction time, or to remember the orientation of a set of geometric shapes.
at least now I understand why you go by 'rms'
I'm saying it's completely bonkers. Are you kidding me? an average IQ of 63? you're talking about a kind of person who needs constant supervision and can barely dress himself. A far FAR more reasonable explanation is that Ethiopia is poor because of political instability, not low IQ.

Note, I'm not saying IQ is a bad measure. In fact from what I've seen, IQ is probably the best measure of ability we have (it's not a coincidence that the average nobel nobel laureate has an IQ of 140.) It correlates nicely with a lot of things you would expect. Although it's still far from a perfect measure.

Anyway, I'm saying the study in that wikipedia article is nonesense on the face of it.
where's that signal to noise essay from pg? I've always viewed standardized tests as a way to approximate intelligence, but they aren't very good ones at that. A search will show a lot of discussions about why.

It's pretty silly to use scores that are partially determined by a given input to prove that some people have that input.

It's circular causality at best and blatant deception at worst.
Yeah, I got hit by the same issue around that time last night. I've been using gridserver for dev purposes, but wouldn't dream of hosting a production app on there.

They're good for something like a blog, though - I believe that was the main idea behind the grid, to survive the Digg effect or something.
The tobacco taxes have been misspent. In San Diego none of the tobacco tax money went to health care. Zero, not one percent of it.
Who actually uses FogBugz? All his blog marketing does is make you think people use it but here in Silicon Valley I've rarely heard anyone use it.
I'm not sure the correlation works that way. Wouldn't you expect a smarter society to, uh, trust people to make their decisions rather than leave it up to insane autocrats?

It's also known that non-biased tests, like remembering a sequence of numbers and reciting it backwards, exhibit the same differences in scores.
The article is bullshit. I didn't read anything after the quote "On all standardized exams, including IQ tests, men score higher than women on mathematical reasoning sections."

That's simply not true. Women are known to score higher (and have since the 60's) on tests of calculation, which is just one example.

To have such a sweeping fallacy stated as a general truth betrays a disgustingly prejudiced intent.

There are volumes more wrong with the whole essay, but really it's not worth my time. If you take any particular statement there, and dig a little deeper, you'll be sickened by what comes out.
Interesting. You're probably a competitor of us. We're working on basically the same thing. ;-)
Well, yeah. I mean, if you were to ask me before I'd heard about the IQ thing, I would have said the same. But new data caused me to reexamine my old assumptions -- why doesn't that work with everyone?

An IQ that low doesn't imply what you think it does. It makes it hard for someone to handle life in a highly regimented society, where not knowing the difference between "Walk" and "Don't Walk" will get you killed in a week. But a person with an IQ of 50 will end up with the intelligence of a nine-year-old, and under some circumstances, you could expect a nine-year-old to, say, dress himself and forage for food.
"Women are known to score higher (and have since the 60's) on tests of calculation, which is just one example."

That's true. Usually, the breakdown is that women are much better at arithmetic and the like, but men are much better at more abstract problems. Which is going to make someone a better EE professor?
...or being on Futurama
I used to tutor junior high and high school kids in physics. In junior high, nearly all the top students were girls. For some reason, in high school, there was this huge switch -- suddenly, most of the girls weren't interested in physics or math. It was weird, but I suspect that it's largely a matter of social pressures.

Luckily, I've had no direct experience with this. I skipped high school.
Cool! I thought so, from Diffle. Toss me an email at palish@gmail.com if you want to compare notes.

P.S. Anyone interested in game development should come hang out with the Gamedev crew at irc.afternet.org, channel #gamedev. They're really smart and they'll help you with any problems you're running into.
You skipped high school? How did that work out?
I disagree, with auctions eBay is having a hard time coming up with more profit every quarter to get people excited and there shares bid up. Skype is well positioned to make money. eBay can afford to sit on it until it does.
Well, we weren't talking about EE professors. But, if we were, we're talking about the top 1% of quantitative ability, in which case there's barely any statistical validity in what is written there at all.

Here's a good piece of data. The Center for Talented Youth at Johns Hopkins has been testing youth (~12 years old) across the country since 1972. One of the first things that they noticed was that of the students scoring above 700 on the math SAT, boys outnumbered girls 13 to 1. Okay, wow.

They have, of course, run the tests every year since then, and now it's down to a 3:1 ratio. This indicates a very, very strong cultural variable. The fact that this isn't even mentioned, again, betrays bias.
If you don't mind sharing: which sleep disorder do you have?
I hadn't heard about that. I imagine it may be cultural, but again, note that the SAT's ability to measure top scorers has been dampened. As far as I know, standard deviation is more significant than absolute difference in the men/women IQ debate. Men swamp women at the high ends because there are more male geniuses and male morons.
Pretty well. I graduated in Computer Science and Physics with a first class honors degree, and the university medal (I was the top student), at 17.

I then went into Princeton to start a PhD in plasma physics -- I did that for two years but that's on hold now -- I'm starting a startup, as I guess is the ambition of many of us here.

Socially, I've always been fairly mature for my age. I got along better with college students than my age peers. It was a very good move. I hated school.
Dear Dani -- i think most people on this board agree with you. I suggest you read: http://paulbuchheit.blogspot.com/2007/03/are-you-my-audience...

to remind yourself that you don't need to expend much effort on this issue :)
There is definitely something cultural to being able to remember sequences of numbers in reverse: I am sure societies that don't have a concept of numbers would have a much harder time with it... As for the geometric shapes, remember the example of the africans building 2d models of the 3d drawing. It seems likely that there is a cultural component to seeing abstract shapes. The Kanji example was perhaps bad - OK, think about recognising kaukasian and chinese faces. They are just faces, no? But somehow it is hard for many kaukasians to tell chinese people apart, and presumably vice versa. I don't think that is genetic, it is trained. It makes me think twice about the cultural aspects of recognising "geometric shapes".

For completeness I want to mention that I don't doubt that IQ tests measure something. Like if they predict success in the military or whatever, they are probably right, ie the IQ63 guy from Ethopia (from another post) would probably have a much harder time in the US military than the IQ 100 American. It is just not clear that it measures "intelligence" per se. And perhaps for the Ethopian military, the results would be reversed. Ie try putting up "surviving in the Saharah without water" as an IQ test.
Do goofy names mean we're in a bubble?
Just resubmitted the application, again, after reading your awesome post. Thanks for sharing your experience.
The thing is that intelligence isn't some kind of nice, statistically normed quantity. There's more to most variables than a mean and a standard deviation -- so I don't know why people seem to always think that you can restrict a discussion of intelligence to such concepts.

One thing that is actually correct is that there's more variation in gene expression between men and women. On of the reasons for this is because, chromosomally, women have two X chromosomes, which pair up, and then randomly one or another turns off. These means that the gene expression is mixed between both parents.

On the other hand, the Y chromosome is largely only effective at turning off gene expression. You can see how there's less redundancy here.

But intelligence isn't a gene. Researchers have, since the time of Dalton, tried to find a simple, biological basis for genius. You know, memory capacity, reaction times, brain size, brain structure convolution, etc. They haven't found anything -- literally everything has turned out to be a false start, even brain size, which, has been shown, within families does not even predict g.

In the mid ranges, there are greater standard deviations, yes. But every single normed test is normed on a sample on the order of 1000. They're designed for regular people. The designer of the Weschler has adamantly opposed the use of IQ tests for anything other than clinical settings, for this reason. It's just no good at drawing conclusions on the extremes of ability.

A better guide might be actual performance. The IOI this year had more girls than ever before -- 11. That's nearly 300% more than last year, where they had 4, and one medal. They are:

Emina Bukva (Bosnia and Herzegovina)
Constanza Contreras (Chile)
Anna Currel (Spain)
Romina Huenchunao (Chile)
Vaiva Imbrasaite (Lithuania)
Taksapaun Kittiakrastien (Thailand) [Silver]
Sepideh Mahabadi (Iran) [Gold!]
Radwa Metwali (Egypt)
Katie O'Mahony (Ireland)
Phitchaya Phothilimthana (Thailand) [Silver]
Ye Wang (USA) [Silver]

Sepideh Mahabadi had one of the best performances of anyone. If you're familiar with IOI scoring, only the top 1/12 are to get golds, and 2/12 to get silvers, thus 1 gold and 3 silvers out of 11 implies that they did at least as good as the boys, and in fact somewhat better.

Were the 'standard deviation' explanation correct they should have, instead, had 0.4 girls earning maybe 0.01 medals. It just doesn't work. Psychometrics are a non-science.
Wow, very nice. Althogh my 'break time' link is gone :(. But, still, I really like it. Thanks a bunch.
Best idea I've heard in a long time. Better hope the smokers don't sue you, though.
Towards Personal Black Boxes: what'll make this innovation happen?
One little feature request. Can you list the threads in descending order by time of visit? This way, the ones I visit last show at the top. I think it can be done if you attach the time-stamp to the cookie, and then order by it.
I tried to implement this into a blog engine I wrote at my last job, but management didn't get it and directed me to stop working on it. I quit to work on my startup a month later.
We're building an easy-to-use tool for making web apps. It's interesting, although we're not competitors, we're both empowering individuals by letting them make something which would ordinarily require programming skills.
Getting banned from places is fun.
Intel Launches a Digg to Rate Software Startups
Internet Stars of 2001: Where Are They Now?
That's my plan. Take what you get and run with it.

I mean, I'm not trying to build a commercial site or anything, but you can bet that I'll take my Google stats and act accordingly.
Yeah, let me look into it.

New version with delete by cookie. http://akkartik.name/newsyc-comment-tracker3.user.js
Do Not Name Drop To VCs
The fact that name dropping is a bad idea should go without saying, but some entrepreneurs have proven the need for this post.When you are presenting your business idea it's a bad idea to rely too heavily on the opinions that big name people will validate the quality of your business...
When Google acquisitions go wrong: the disappointing story of Urchin
"But intelligence isn't a gene. Researchers have, since the time of Dalton, tried to find a simple, biological basis for genius. You know, memory capacity, reaction times, brain size, brain structure convolution, etc. They haven't found anything -- literally everything has turned out to be a false start, even brain size, which, has been shown, within families does not even predict g."

I hadn't heard that about brain size. The correlation I read about was 40%, over a number of studies. It would be interesting to know if this had been updated, since I know individual cases belie it. But that's a small sample size (like your IOI numbers -- 11 people in one year is extraordinary, unless it's an outlier).

But even if those numbers don't predict g, you're, uh, left explaining how g varies when what it's made up of doesn't vary. You're implying a correlation between any two intellectual activities (e.g. counting and calculus, or reading comprehension and analogies) of zero. Unless your sample size has five idiot savants for every Gauss, I just don't see how that's possible.

"Psychometrics are a non-science."

How do you account for the explanatory power of IQ, then? Is it a myth we've all bought into? Is it suggestion?
I bet google does'nt want a rails app.
Can't recommend them.

Been with them for a month, major outage earlier this month for two days.

They seem good for blogs, or staging, but if you require reliability, go somewhere else.
Also, notwithstanding the fact that this is a tech/science/math forum, whos to say that aptitude in math/science is a more equivalent measure of intelligence than it's antithesis e.g. art,juggling talent, etc
Interesting, how is the "abstractness" measured? Isn't arithmetic rather abstract?
Testing Myth: TDD Not Suitable For Exploratory Programming
http://en.wikipedia.org/wiki/Non_24-hour_sleep-wake_syndrome

http://en.wikipedia.org/wiki/Delayed_sleep_phase_syndrome

http://en.wikipedia.org/wiki/Sleep_apnea

My symptoms are exacerbated by a couple other health issues, and the doctors have had difficulty establishing causal relationships. I use a CPAP for the apnea.

I feel like I have jet lag if I miss a little sleep. If I miss a great deal of sleep, my symptoms worsen dramatically...I had a stroke when I was 30 after being sleep deprived for too long, for example.

I was able to do all-nighters without a problem until I was a senior in college. Then, I worked night shift for seven years to make it easier to cope. The sleep issue was yet another motivator for starting my own company.
http://en.wikipedia.org/wiki/Chinese_character
You might want to collaborate with some people who are building existing lexicographical infrastructures. Have a look at this TED talk: http://www.ted.com/index.php/talks/view/id/161 . You might also want to check out E. O. Wilson's Encyclopedia of Life at http://www.eol.org/ . (He's also got a TED talk where he won the TED prize for the year.) Both of these people are doing grand, sweeping thinking about ways to use the web to redefine the way we think and use language, pictures and taxonomy, all of which you'll need to think about as you're developing this idea. You might see if you can eventually get some phone time with them and see how they solved similar problems you may encounter, like a structure for crowdsourcing. I'd love to see how this idea develops! I'll be very interested to see how you illustrate abstractions. Good luck!
Um, no-brainer: if you sell something people can use can on their own servers, you can't spy on their logs. Everything Google does is geared towards collecting data, so the other Urchin products are really not relevant for Google.
Why would a brilliant autocrat decide to trust the people's decisions?

The political and economic freedom of Western Europe and its friends is
substantially the result of historical accident -- as is the poor state of
Africa. Many of the post-colonial nations experienced autocratic catastrophe,
seemingly in proportion to the degree in which industrialism was a jump for
them.

If you want tell me what ethnic group you are descended from, I would be happy
to tell you how stupid you are based on some unhappy incident in your people's
history. I, for one, have a lot of German in me. Are Germans smart for their
rockets or dumb for their Fuhrer?
I remember reading about a startup in Israel doing exactly what you are describing. Their product is positioned as an alternative language for faster/easier texting on cell phones.

(if I remember correctly)
Here: http://www.zlango.com
>>"Psychometrics are a non-science."

>How do you account for the explanatory power of IQ, then? Is it a myth we've all bought into? Is it suggestion?

The IQ test means a lot in the midrange in western cultures, but becomes completely worthless at the extremes. It's not so much that there is no scientific basis behind psychometrics as it is just the beginnings of a science. It's really hard to understand the brain, I'm hopeful that we or our AI overlords will get there eventually.

And Dani: I have no doubt there is a genetic basis for intelligence, we just haven't figured it out yet. Clearly it is a whole lot more complicated than a single genetic mutation. This one gene has an influence on signaling pathways in the brain which has an indirect influence on intelligence. http://www.physorg.com/news91799494.html

Also see the disproportionate number of Chess Grandmasters and Science Nobel Prize winners among Eastern European Jews.

Summary: http://www.nytimes.com/2005/06/03/science/03gene.html?_r=1&#...

Paper: http://homepage.mac.com/harpend/.Public/AshkenaziIQ.jbiosocs...
I know a few smart women. Even if there are important statistical differences between men and women in the way of analytical ability, we still have to take individual men and women on their own merits.

It's too bad that colleges are unfairly admitting sub-par women, but then again, that's probably not as bad a thing as admitting all those god damn business majors...
"Brain size does not predict general cognitive ability within families"

http://www.pnas.org/cgi/content/abstract/97/9/4932

You can't discount that IOI statistic becuase it's an outlier. Every single participant at the IOI is an outlier in cognitive ability. Do you know about the theory of outliers?

There's this thing called the central limit theorem. It says that if you have a lot of small independent variables, randomly assigned some value, then the mean of all these variables (or, by the same token, the sum of the variables) is distributed approximately normally. But suppose the variables are not small, or they're not independent. Then the central limit theorem doesn't hold, and what you have, almost all of the time, is an outlier -- that's why there are often many more outliers than you'd predict in a given population, using a small sample.

Now, I'm not saying that g is zero. I said that psychometrics is a non-science, in the same sense that a lot of the social sciences are non-sciences (you can find papers which try to show a causal effect of insurance regulation on premium prices, ignoring profits entirely, for example).

The fact that g is non-zero can be readily explained by the following simple observation -- most academic subtests, including IQ's, rely on skills that are either practiced as a group, or on skills that are shared between subtests. One example is focus, in general. Another is visualisation. Another is working memory. And so on and so on.

Many of these skills are also practiced in situations, like school, where if one does well in one area, they do well in another. If you're the teacher's pet, you get more attention. If you're known as the bad kid, you're immediately discounted (and I've been on both sides). If you're poisoned against a learning environment, you just won't put any effort in.

So it's no mystery to me that g is non-zero. The point is that the field of psychometrics is totally absent of content. There's no objective test for the validity of a test, for example -- the best they have is g-loading. Over the years, this means that tests have become higher and higher g-loaded. Now this could mean that the tests are getting better, or it could be that the subtests only look different, they are becoming more similar in content.

I've been studying these tests, the actual tests, since I was twelve. It didn't take long before I figured out how poor they were at answering research questions, or questions of individual ability. If you get the chance, try to look up the history of the Stanford-Binet, or Terman's kids, or actually take a look at the scoring method behind most of these things. They're totally full of crap...
"Snowdrift" game tops "Prisoner's Dilemma" in explaining cooperation
my point is that they should be one system. Similar to how email and chat are the same in gmail.
I'll be switching to the Miss America Kid-Safe browser
I'm going to try and use it exclusively for a month and I'll let you know how it works out.
Wow, Dani. Impressive. Need a co-founder? Any workers?
Finding and Evaluating Deals Online
For the cash strapped startup (or starving student), it is often important to find the best value at the lowest cost. Here is a collection of web sites that can help find and evaluate deals. Where do you find and evaluate your deals?
Ask YC: What are the likely server costs for a facebook ad network?
Ask YC: What are the likely monthly server costs for a facebook app that serves the facebook developer community, i.e about 5000 apps and rising? Assume that the app is an ad network of sorts and does not need to store video or audio media files. Thanks.
rms,

Yes, I agree with you there. It's just there are constantly pronouncements that something or another has been found, and the true story always seems to be that it makes no real difference. Certainly there must be a genetic basis, I just have real doubts that the standard explanations of genetic variation can actually account for it. I mean, say, hypothetically, that it took 15 genes to produce genius. Then the lack of redundancy and higher variability on the male genotype would be a hindrance, not an advantage.

"...see the disproportionate number of Chess Grandmasters and Science Nobel Prize winners among Eastern European Jews."

If you plot said statistic over time, even taking the Jewish population versus the rest of the world into account, you'll notice massive fluctuations. Anand just took #1, for example. I bet you'll see a lot more young Indian chess players. In different times you could say that the world's intellectual leaders were Sumerian, Babylonian, Indian, Chinese, Greek, Roman, Mayan, Arab, Milanese, Viking, Khmer, Maori, Catholic, Protestant, Lutheran, Jewish, Russian, or American. Don't forget that the Nobel prize committee are themselves Nobel prize winners, and they have not shown themselves to be free of bias (They didn't award Rosalind Franklin, nor Emmy Noether, nor Freeman Dyson, and I'm sure, many, many others). I also think that the selection of Chess and Scientific Nobel prizes is clearly occidental. Why not Go (Japanese/Korean) and Tang (Chinese) era scientific achievements? Why not programming competition performance (Poland and Russia seem to be leading) or memorization of oral history (The Tibetans seem to be quite remarkable in this respect).

Or for that matter, the first masters of Astronomy (Babylon, Mayan) or masters of the Mesoamerican ballgame (Mayan, Aztec)?