skysight-inc/HackerNews huggingface dataset:
I'm sorry, I majored in engineering rather than Real Science (TM) so I might have missed this lecture, but when I'm doing stats I don't start with a data set, a faked version of The Right Answer (TM), and a list of adjustments to try prior to finding the proper one (i.e. the one which results in The Right Answer).

Though now that you mention it, that seems like it would save a heck of a lot of work.

Surely if you present a chart, you also have to present the way you derived it, at which point any fakery should become evident.

If nothing else, this scandal is exposing the difference between the dogmatic version of science we all learned in the Scientific Method lecture back in 6th grade, and science as it is actually practiced in the real world. The Scientific Method lecture says that all results must be reproduced by experts. In the real world, the computer programs involved are outrageous hacks, the data are practically never seen after publication (and go missing with disturbing regularity), and the peer review concentrates mostly on making sure you cite the right people and scratch the right backs. Nobody is ever going to try to actually duplicate the math it took to transform your raw data into your massaged data -- that is hard work and have you ever met an engineer who would reliably leave detailed instructions to exactly reproduce 6 months of labor, in a form where they would be instantly comprehensible to anyone in the field? I would be astounded if scientists routinely actually committed to paper an exhaustive list of all the software installed on the computer they did the math on. (What version of which library, etc etc. So you can reproduce bugs.)

Then there is the "Frustrate anyone who opposes us at any turn, destroy data in response to FOI requests, get our nemeses expelled from their institutions, and ruin journals which publish competing work" angle, which takes it from "science as it is practiced in the real world" to just farce.
It's much more fallacious to discount the overwhelming scientific consensus on global warming because of one scientist's character.

A virtual army of teabaggers combing over this data and all they could find was one marginal case of cooking the data? I'm more confident than before.

Also -- it's not fallacious to point out historical bias by the author when discussing a partisan issue. It's additional information to be taken into account.

EDIT: Partisan downrate? Please, people, go to redstate.com or something if that's how you feel.
In 20 years, there is going to be a wave of concern that we aren't praising our kids enough.
Sorry I missed how you mean a flaw in the proof. The thing about evolution theory is that evolution is a stochastic process. So there tend to be not precise predictions, just likelihoods. Also, it is being applied in thousands (or millions) of cases to explain complex systems. If sometimes somebody who applies it gets it wrong (like say some biologist trying to make sense of bone findings and creating a theory of the evolution of some species or other), it does not refute the whole of evolution theory. It only refutes that particular application.

It is like physics: millions of buildings have been built employing Newtons laws. Sometimes an architect gets it wrong and the building falls down. But that doesn't refute the laws of physics - it is just some guy or a team of builders making an error.
I think a better formulation is "I don't listen to people who've shown that they're not going to argue in good faith". If ESR has really been in deep teabagger territory, then the fact that he's smart really has nothing to do with it -- he's not seeking truth, he's seeking confirmation, so who cares about his conclusions.
How's that old quote go? Something about winning by not playing?
I don't want to defend the scientific system, I am pretty sure it has a lot of flaws. However, the papers you mention that only depend on back scratching in my opinion are simply very weak. I am not even interested in discussing such papers.

If anything, the blame should then go to the state departments responsible for funding the research. If they are being blinded by such crap, it is a problem.

That said, I think succeeding in academia requires good skills in politics, I have a lot of respect for some people who manage to do well both in research and in pulling the right strings.
I don't recall claiming it was.

I simply used an example most people here would be familiar with; my broader point remains unchanged for data analysis. One can legitimately critique an analysis of data without providing new data or even performing a correct analysis.
I await the explanation of this little detail.
Ask HN: GRE Subject Test Math
Has anyone on here studied for the Math GRE. If so, any suggestion of material to study from or what to focus on. Thanks.
http://online.wsj.com/article/SB125883405294859215.html

Phil Jones, the director of the East Anglia climate center, suggested to climate scientist Michael Mann of Penn State University that skeptics' research was unwelcome: We "will keep them out somehow -- even if we have to redefine what the peer-review literature is!"
Indeed. And related to efficient data structures.
I've participated in 3 pitching competitions at Edinburgh University. I won one of them and lost at the other two. The value I gained from them came in 3 categories:

1.) Networking
2.) Awareness among local startup scene
3.) MacBook Air ;P

Seriously though, the contacts I've made at these events have been infinitely more valuable than the cash and assortment of juicy Apple products that I've won.
The whole idea of a business plan contest is screwy. Ideas are a dime a dozen, it's all about execution. If I were to start another startup tomorrow, I'd just scrape TC for the newest and hottest set of startups for an idea, tweak with my own twist and attack head first.
Yes. It's super easy- a 780 is 90th percentile, and an 800 is 94th percentile.

The best thing you really can do are practice tests. Just to warn you though, the Kaplan etc. pre-tests are actually much more difficult than their regular practice tests, and the regular practice tests are more difficult than the regular exam.

This way, you will always do better on the actual exam than on the practice tests, and this will make you believe that studying helped, which is actually doesn't.

The only accurate measure of how well you'll do are the practice tests given to you by the actual GRE people, and you'll get a disk of those a month or so before your exam, in the mail.
Good analysis and I agree with you on using custdev as a tool in your tool kit rather than follow it like a religion. Another point, Steve always talks about shortcutting the custdev process if you happen to be the SME on the subject. If you know lot about what customers want in a area because you have experience in then you can skip portions of it.
Creativity is Not Design - Design Test 2
Right, I'm going to wade in here. Tichy I completely agree with you, the evidence exposed by these leaks is very, very weak.

Basically there is already existent a lot of evidence that many scientists disagree strongly about Climate Change. People who have doubts should not need to lower their arguments to a shrill 'Aha I see a smoking-gun'.

This morsel of evidence is tiny by comparison to more eminent disagreement.

---

For more weighty disagreements see amongst many more:

S McIntyre and R McKitrick 'Corrections to the Mann et a. (1998) proxy data base and northern hemishpere average temperature series' [1]

as Nigel Lawson[2] says: 'it turned out ... that crucial data for the period before 1421 was based on.. one single pine tree'

-----

[1] http://www.uoguelph.ca/~rmckitri/research/trc.html

[2] p126 note 39 of A Call to Reason: A Cool Look at Global Warming by Nigel Lawson. The 2009 release has lots of up to date information in a foreword
Well, an average person manages to pay taxes, live a reasonable life and leave an inheritance. So I'd say, yes, he makes a positive input.

An over-supply of labor is pretty good for the economy as a whole, bad as it is for the unemployed. We're not even close to the point where the over-supply is so big it causes real problems. There are still areas where there is a labor shortage, especially in skilled professions.
I'm not disagreeing with that. AGW may be true while the corrections to this data series may be false, and this data series may be only a small piece to the puzzle.

However, fraud and errors are different. Science is (to a great extend) based on trust. When I referee a paper, I trust that Fig 1 is really generated by Algorithm 2 if the caption says so. If it turns out that Fig 1 is photoshopped, my report is meaningless. Peer review is meant to find mistakes and evaluate importance, not eliminate fraud.

If a scientist cheats, we must consider all their work to be flawed until they can prove otherwise (i.e., release all data, code, etc). We need to stop and scrutinize all the fruit of their poisoned tree.

AGW may be true, and the science may be solid. But until we have more openness, we will never really know.
It needs about one more year to pickle. It's way too young.
Also, Jon Burgstone came in second place for the HBS business plan competition with his idea for SupplierMarket and then started and sold the company for over a billion dollars within a year. Now, he and his former co-founder offer a special (and larger) cash prize to other 2nd place participants of the same contest.
I belong to the BBB so that I can have their logo on my site. I thought it was expensive since that is all I want of them.

It frustrates me that scammers such as Video Professor use the service.
A great example of what's wrong with the BBB, which "rates" businesses that pay them fees.
Black Friday - Are you REALLY saving that much?
Interesting to see an analysis on whether the deals on Black Friday are actually good. As you would expect, some are very good deals but some aren't...

That $67 Mac mini from Amazon was a good deal though! Damn sorry I had meetings all day and didn't get to a computer fast enough!
clojure, scala, jruby, etc all can use java's large source of libraries.
It's like a game of Civ come to life.
A shower does more than just kill micro-organisms. It might be useful as an extra step after a shower but there is nothing to suggest it would be effective at cleaning off the dirt accumulated during a day.
The 00's: A Decade from Hell
As someone who is very close to both of the topics he's discussing (bootstrapping and customer development), I'd like to point out something obvious that the author has apparently missed: they're not the same thing. You can raise money and do customer development, and you can bootstrap with your head in the sand and never talk to your potential customers.

The value of customer development is that it's a model for being more efficient with whatever resources you have as you search for a fit between your product and the market.
Reactive Extensions for .NET (Rx)
Link to first page: http://www.time.com/time/nation/article/0,8599,1942834-1,00....
The only thing that's a "big deal" about that is the laziness and mendacity of its author. The data he found was published in 1998 and 2000, a fact he mentions without irony while arguing that the data was "hidden".

The linked article also claims that the NOAA "deleted" the post-1960 data from a data set (briffa2001jgr3.txt) put together for an article that doesn't use these data. In other words, what really happened, is that Briffa and his collaborators created a data set only containing the data they used, and this derived data set was archived by NOAA. That's hardly nefarious or even the least bit worrisome.

Now to the heart of the issue: Why was the post-1960 not used after 1998/2000? Why do all of the e-mails talk about "hiding the decline" and the code talk about stopping in 1960 to "avoid the decline"? The answer, it turns out, is right in the very Briffa 1998 paper your link mentions (but fails to cite). It's title is Reduced sensitivity of recent tree growth to temperature at high northern latitudes and it's about how temperatures reconstructed using Briffa's MXD (maximum latewood density) data diverge from actual temperature readings after 1960. The implication of the word "reconstructed" is very important to understand: Briffa's MXD data don't show an actual decline in temperature. They show a decline in a particular measurement of tree-ring density that had, until 1960, been a good indicator of actual temperature. All that's going on here is that data that is known to be inaccurate is not being used.

It's no big deal.
I've always found yawning very intriguing, and how people often get triggered to yawn as others around them are yawning. And how I just yawned at the thought of that!
In what way can it reasonably be said that the inventors of Go also invented regular expressions?
Not that scammy? Nowhere on the site are any of the extra charges declared. If someone bills your credit card without expressly telling you that they're going to then it's a scam.

Are you seriously trying to tell us that tricking unwary customers is not scammy?
In the case of

http://en.wikipedia.org/wiki/O3b_Networks,_Ltd.

HN does it wrong. There are exceptions either way, I wouldn't say that one is more correct.
There actually is a statistically significant difference between .6 and .9 with those sample sizes. So the conclusion should in fact say that there is a statistical significance.
You do not need to know calculus to run a LBO model. Writing one is easy enough and most of the time the firm you are with has a few dozen variants that have been prebuilt for you. The analytical portion of PE isn't the hard part.
See spoondan's reply.
Exactly, there is no incentive to hand down bad ratings to the people giving you all your money. Way OT but lack of proper incentive is a huge problem everywhere and played a major part in the banking collapse.
Your points are well taken, but many of us get information from trusted blog sources and Wikipedia, and hopefully we are discerning enough to be able to look at multiple sources, sometimes to engage in discussion, etc.

Now I like reading good news sources as much as the other person (the Britannica I will probably never read again unless a search takes me there), but the way to consume and find information has fundamentally changed for a large portion of the populace. Not sure if it necessarily means that news necessarily gets outsourced, but definitely gets fragmented and distributed. The key for news organizations (as opposed to a newspaper which is a medium) is to figure out how to stay relevant in that environment.
A very thorough argument, albeit with a trollish/link-bait title. Also, please note that the author maintains a Lisp implementation (CLISP) and might have an axe to grind; generally, never take a Lisper's opinion on an ML dialect, or an MLer's on Lisp; the two groups are mutually hostile and philosophically incompatible (kinda like Skinheads vs Trekkies)
I did, and it worked :) Really feel energized now :)
http://www.tradekay.com

We are wholesaler of Nike Jordan and Other Shoes. We are a professional exporting company . We supply many kinds of Shoes, such as Nike Shoes, Jordan 1-23, Air Jordan, AF1, DUNK, Air max series etc. Most of them are in stock and can be supplied surely on time. All these shoes are packed with original-boxes and cards.

AND We do wholesale and retail! All are extremely CHEAP, please visit:

http://www.tradekay.com/productlist.asp?id=s1 (UGG)

http://www.tradekay.com/productlist.asp?id=n506 (JACKET)

http://www.tradekay.com/productlist.asp?id=s3 (NBA Jersey)

http://www.tradekay.com/productlist.asp?id=s12 (JORDAN SHOES)

http://www.tradekay.com/productlist.asp?id=s70 (TSHIRTS)

http://www.tradekay.com/productlist.asp?id=s75 (JEANS)

http://www.tradekay.com/productlist.asp?id=s69 (HANDBAGS)

http://www.tradekay.com/productlist.asp?id=s57 (sunglass)
Managing Technology Means Being Wrong a lot
I wouldn't say "not possible". Interoperability between java and groovy is quite good. It's to some degree true for clojure and scala -- two languages I haven't used myself yet. There are example snippets around in the Internet that demonstrate that a combination of groovy/scala/clojure/jruby is possible. So the situation isn't that bad.
Your "obvious" rules are as arbitrary as those of any other language.

What is the difference between a tuple and a list that's "obvious" to a novice programmer? (Yes, you may claim that I change my argument to talk about lists versus tuples, but if you want to press on this line of thinking, Python's syntax for list and list-of-list declaration is the same as Perl's syntax for anonymous array declaration.)

Which parts of the requirement for a trailing comma after the first element of a single-element tuple are "obvious" to a novice programmer?

Which of those "obvious" rules are obvious cognates with other programming languages?

(What's "obvious" about Smalltalk having a strict left-to-right evaluation order, ignoring "obvious" mathematical precedence rules which schoolchildren learn?)

> ... in Perl, you've got the same expression that creates a list in one case, but not in the other...

This is still not true. The comma operator creates lists in Perl.
Let's hope the kid stays asleep in 5 hour blocks, because that's how long my hacking runs go.

Programming doesn't suit people who are constantly distracted; the way I cope with others' presence around me is to completely ignore them. I can have conversations for hours on end while I program and not remember any of it. I agree to things, laugh with jokes and listen attentively. All of it a facade, of course; I don't really care about others when hacking.

Days run into each other sometimes and you have no idea what went in the world. I wouldn't do this when I have a baby, and I don't recommend it to new parents. Do marketing for 2-3 years, instead, and give that kid full attention when you go home.
"...after writing my column I’ve received a few emails from readers who have themselves made the switch from finance to manufacturing."

Isn't it tradition for those bored of making mega-money in the City to retire into "manufacturing" - the manufacturing in question being invariably that of bourgeois foodstuffs? Organic smoothies; organic hand-reared pig farms; organic gin; organic English sparkling white wine; organic unpasteurized cheese; organic luxury takeaway services; etc etc. Either that or they'll snap up a few struggling restaurants and learn how to cook. It's like they feel dirty after handling all that grubby money and need to get their hands back in the fertile soil to nourish their withered souls.
In any case, this all seems to be very ad hominem (revisionists trying to discredit scientists instead of providing data)

Yep. That's a great summary of the contents of the emails.
This article makes me very happy that I'm currently learning F# instead of plain vanilla OCaml.

Having said that, some of the code to me just doesn't look right. Like the file open nonsense. Why not just write a generic function and use that whenever you need it? Do you really have to repeat all that nonsense every time? Because if you're basically copying and pasting, maybe you're doing something wrong?

I don't know, since I'm a newbie and I'm on F#, not straight OCaml. I know in F# I can get an open file (with error catching) in just a line or so of code.
Using game theory to predict the future
Being right doesn't make it interesting or well written or worth while reading.
No, you are misinformed. Further, your tone is unhelpful.

This was the first time that the data was available. As McIntyre points out in the top of his post, the decline was discussed in the 1998 and 2001 Briffa papers.

Of course, scientists could not actually evaluate this because the data was not released. But that's not the issue.

The issue is, as McIntyre also points out, that the IPCC 2001 graph (and later graphs) simply truncated the climate reconstruction data in 1960 and pasted the (more accurate) direct temperature station measurement data onto the graph after 1960.

While this was done it plain sight, it's still lousy scientific practice. Why? After all, shouldn't more accurate data be used? The problem is that comparison of temperature reconstructions with direct measurement tells you a lot about the accuracy of your reconstruction.

In this case it tells you a lot about the inaccuracy of the reconstruction and the sorts of signals that are missed.

I'm sorry, the paleoclimate reconstructions available in the literature are not convincing. They often use dodgy statistical techniques, mixing data gathering steps with data correlation steps to increase apparent agreement with the instrument record. There have been huge problems with releasing data and there are often bizarre choices made in proxy record selection.

Without good paleoclimate reconstructions, it's hard to know how far outside (or inside) the historical standard deviation for temperature we are. I happen to think it's an important question to have answered. A trillion dollar question.
Noobcomments View Allows Flagging Spam Comments
Yes OpenID it does provide your email address. That's why my gravatar icon shows up wherever I login with OpenID.

You don't need to type in a long URL to login using OpenID. If you have a domain you can use it to delegate the OpenID provider to use for logins. So you need something like openid.mydomain.com and a username and login. One set of authentication information to remember instead of different ones for every website or application you use.

You wouldn't want to use Webkit to implement OpenID on the iPhone anyway. You would want to integrate it into your application so that was a seamless process.
How do we know that post 1960 tree ring data is bad, and needs to be adjusted, yet pre 1960 data is good? What happened to tree growth after 1960?
Doesn't look like anything more than a sexp reader; very limited (I am guessing a 1-2 day hack.)

All it has are: atom, eq, car, cdr, cons, cond, lambda, define (for variables!) and defun.
It's also an interesting view into what topics and comments motivate people into getting an account and participating.
For what it's worth - I'm a professional lisp dev who greatly admires the ML family and its ilk (Haskell, Miranda, etc).

I'm not very good with them (yet), but I can appreciate the safety guarantees of a good type system (as long as it has decent inference and can avoid the ridiculous verbosity of Java).

As a rule of thumb, if I want to hack something up quickly, and I use Lisp (or python/perl). If I want something safer, easier to maintain (not necessarily extend though ...), etc I like languages with stronger typing and more powerful static analysis.
I'm still hoping for a Space Race Victory...
This is reasoning that programmers can be mothers not that mothers should be programmers.

But one main problem is when children are young mothers are much more likely to be tired or worn out running on little sleep and probably have less time/will to work on complicated problems
None of these articles even attempt to mention culture and upbringing, which plays an immense role. For example, the person in the article came to the US when he was 7. He is most likely very much Americanized and would feel a foreigner in India, whereas 20-something Indians that come to the US to do their masters/PhD probably can't wait to get back, where they can feel at right at home.
Huh? Where are camel cases used in German?
> Nowhere on the site are any of the extra charges declared.

Be careful of what you are claiming. The http://videoprofessor.com homepage says "After your 10-day free trial, if you decide to keep the complete set, we'll conveniently bill your credit card just $289.95."
He is bashing two strawmen:

1. Lean Startups aren't cheap startups

http://steveblank.com/2009/11/02/lean-startups-aren’t-cheap-...

2. Lean Startups aren't small startups.

Steve Blank based much of his methodology on his experiences running E.Piphany. E.Piphany IPO'd before reaching a 4 Billion dollar market cap:

http://news.cnet.com/IPO-Update-Kana-Communications,-E.pipha...