skysight-inc/HackerNews huggingface dataset:
There wasn't much to go wrong... The A/B testing system renders one template in two ways depending on whether the user is in the test or control group, and the template was the same in both cases. If we had made a mistake, the results wouldn't normalise when we got more data, they would have just stayed there.
"But only after reading Paul Graham's articles on Wealth, I realized why making a fortune from poker always seemed "empty" to me: you're not really creating wealth that way."

You needed to read PG's essays to figure that out? Gee, no wonder some people joke that HN and YC look like a cult at times. Do all your thoughts on wealth-creation stem from PG's gospel?
FIRST and REST are from Common Lisp.

http://www.lispworks.com/documentation/HyperSpec/Body/f_firs...

http://www.lispworks.com/documentation/HyperSpec/Body/f_rest...
Not necessarily. Yes, there are many Christian, Muslim and Jewish scientists who are practising believers. However, there are variations in practising belief. The type of thinking in the lab is not the type of thinking done in church. Now this doesn't necessarily stop scientific progress from occurring in and of itself, but it slows it down significantly. Neil deGrasse Tyson explains this very well here: http://www.youtube.com/watch?v=0vrpPPV_yPY

The second way is a political conflict, like Galileo having to say that the Earth was the centre of the solar system. Stem-cell research being discouraged in the States, and a number of other issues faith/the religious have with areas science reaches into.

Another thing this article is saying is that religion and faith as a way of discovering or deciding on truth and morality are waning as we move into the future.
> You do not create wealth - you make another poor soul, likely a gambling addict, further burdened by that addiction. Meanwhile, the only person you're "creating wealth" for is the guy with his feet up that owns the poker room.

Does a sportsman make wealth? What about a professional chess player? Snooker player? Pool hustler?

What about a brewery, "further burdening that addiction" of alcoholism? What about the barman what served the drink? Just creating wealth for the pub? The sommelier?

What if I start a start-up that's about 'social' betting, like a betfair exchange about film stars? What if I shake up the gambling industry with my social facebook Zynga poker that launches real-money tables?

Your concept of what is a 'valued' job seems very narrow.

Gambling is an enjoyable past-time for millions, throughout history, in cultures around the world. For every sad story of gambling addition, that are thousands who enjoy gambling responsibly on a recreational basis, just for every alcoholic there are thousands that enjoy drinking socially. And some social gamblers sometimes blow a bit too much, just like social drinkers sometimes go overboard at the weekends. Things aren't black and white.

And those professionals provide the entertainment that others enjoy, either directly (from spectators on TV) or indirectly (providing that example that 'the game can be beat').

Sure it's not the most worthwhile job in the world in the grand scheme of directly creating wealth, or improving man-kind. But then, dare I say it, nor do do many a start-up.
Wall Street reminds me of poker done on a much larger scale.
Sure, but the way you estimate your sample size is working backwards from the confidence level you want to get. In this case, we were waiting for the confidence metric to tell us when the sample size was appropriate, but it didn't work out as planned.
This is really cool. I know a guy who used MS Flight Simulator to test the control algorithms he developed during his PhD. Looking at MATLAB plots does the trick, but seeing a virtual aircraft being stabilized after entering a forced flat-spin is immensely satisfying.
I'm skeptical that merely privatizing NASA or really any other public sector agency would automatically remove bureaucratic fluff, and that bureaucratic fluff is such a gargantuan problem at NASA that removing it would cause a miraculous internal shift for the better. It strikes me as more of a libertarian utopian cure-all than a decision based on NASA's particular problems and objectives.
Ask HN: Need Contacts: small teams which use multiple online services
Hi All:

We are working on a product where our users are small teams, freelancers, and consultants which use multiple online services (Google Apps, Basecamp, Highrise, Skype, Odesk, etc.).
We have heard (and have experienced by ourselves while developing www.showzey.com) that it is a challenge to search and sync/transfer documents, contacts, and communications stored in different online services such as Google Apps, Skype, Basecamp, and others.
Our goal is to make it super easy for small teams or individuals to integrate these services into their work-flow.

Would any of you be willing to speak with me for 10-15 minutes? I promise to not take too much of your time.<p>We are not selling anything - but merely looking for customer insights.

And, of course, in return, I'd love to see how I can help you as well.

Thanks a lot,
Senad
The worst part was that we actually did make decisions based on the confidence metric when it was as low as 95%, sometimes, and then realised that we shouldn't have (we should have waited for more data).

This is what most people who talk about A/B testing don't mention, that you need more data than you think, in order to get good results.
All About Skimmers
Considering they have not released the full android version, and the fact that they can't use any of their existing code -- quite a while. I also think they can't use any of the open source libraries they used on iPhone, Android , etc... because they have to use XNA.

If i were them, I'd wait untill Microsoft adds native support. By the time they finish their game using XNA they might have released native support, and then they're stuck with 2 codebases.
So you would lose money by staying a programmer? :)

Expenses seem to be on the higher side. 400$ is okay for a simple lifestyle.
Well, they rebrand SheevaPlugs as TonidoPlugs and install their cloud suite of software on them. (We host both at Pluggr)
Your ridiculous talking points are well overplayed. Let us start with the e-mails - point out where they were "damning" and where information was "falsified", hint: it didn't happen, you're a fish swallowing a hook.
I accidentally put a USB stick in the laundry once – it worked perfectly well, despite being in hot soapy water for an hour.
Anecdotal counterpoint: I have two buddies, aged about 25, who swear by Star Trek, especially the original series. Don't know why.
"97 percent of all climatologists worldwide __assume__ that greenhouse gases produced by humans are warming the Earth"
They are compartmentalizing their belief system. Whatever they believe about God does not propagate through the whole belief system. It is only local.

It is like me updating Firefox and then don't brother to upgrade everything else. As a result, Firefox work and play in a sub-optimal environment.
You can get up to a TB RAID1'd
the previously accepted statistical threshold of "scientific truth"

This concept heartily amuses me. Care to say more?
There were several long discussions about this on this very site at the time, and many of us remember those discussions, so it's going to be difficult to spread FUD about them.

Here's a pointer to a bunch of them, in case you missed the whole debacle: http://www.google.com/search?q=site%3Anews.ycombinator.com+c...
Every time you do a significance calculation and decide whether to stop the test or continue you increase the likelihood of a type I error (i.e., false positive).

So, your 99.8% confidence isn't 99.8%.

There are a few ways to compensate for this. The easiest is to fix your sample size and not reach any conclusions until you have tested that many users.

You can determine your sample size by, say, figuring out the minimum number of people needed before there's a 95% chance you observe a 10% effect size.

This is a problem in clinical trials where ethical questions arise. If the test appears harmful, should we stop? If it appears beneficial, isn't it unethical to deprive the control group of the treatment?

Anyhow, if you want to observe the results continuously, you need to use a technique like alpha spending, sequential experimental design, or Bayesean experimental design.

TL;DR: If you're periodically looking at your A/B testing results and deciding whether to continue or stop, you're doing it wrong and your significance level is lower than you think it is.
Why have personalized news startups failed?
No, you're doing it wrong.

You say, "How many people should I test before there's a 95% chance I observe a 10% effect size?"

That's your sample size. It's easy to compute up front.
Bloomberg Plans a Data Service on the Business of Government
Science and religion may not be friends, but at least they're not as lonely as philosophy, which seems to have been forgotten entirely by article writers such as this one.
I would also bet that those teachers/professors are working on some lucrative (research) projects or are members of a board of directors, e.g. John L. Hennessy at Google[1].

[1] http://www.google.com/corporate/execs.html#johnhennessy
They were probably taught their religion before they learned about science. Also, people are good at compartmentalizing beliefs.
The underlying answer is that SW, also Dune, LoTR, even Harry Potter, create sophisticated alternate universes.

Those universes have self-consistent rules that are quite elaborate, languages, but also, of course, touch and evoke common mythological motifs:

* good vs evil
* father vs son
* sacrifice of self for the good of others
* romantic love
* coming of age
* the journey of the hero
* the faithful friend
* revenge
* supernatural adversaries

In other words it combines good, well established literary motifs with an intricate, self-consistent universe. It lets geeks and nerds with their well developed imagination completely escape the reality into this universe.
> According to a US study, 97 percent of all climatologists worldwide assume that greenhouse gases produced by humans are warming the Earth.

This is a lot like saying "97 percent of all feminists believe that patriarchal oppression is holding women back."

You don't go into climatology unless you think it's a big deal. See, I think global warming (1) exists, (2) is a mix of man made and natural cycles, (3) is not a big deal at all. Apocalyptic global warming just seems obviously false based on everything I've looked at.

Now, perhaps I'm mistaken. I didn't make this comment to argue the case - what I wanted is to point out that because my layman's understanding is that it's not a big deal, I don't go into the field. You'd expect anyone in the field to think it's a big deal.
I think it was a meta-analysis (a study using multiple original studies) and I recall 95% being a number of significance. As in it was either the previous or new, improved and of course lower threshold of "truth" (or rather statistical significance or whatever). I'm going by a 2nd or 3rd hand report, but the reported details are simple and easily falsifiable.

(ADDED: Or maybe it's 90% and they used 85%??? Anyway, that should give you enough to search on if you want to follow it up.)

Epidemiology is hard. When it's about a politicized topic (and almost all are nowadays) I've learned to just not pay much attention to any one study, at least until others have a chance to tear it apart.
The worst things to come out of those emails were scientist getting pissed off about idiots who kept requesting data they were legally obligated not to provide (it was proprietary, if the people wanted certain data they had to buy it themselves), the rest of the data was already freely available. If you believe anything worse than that occurred then I'm sorry you're an idiot.
any relation of the number of increased signups to the Xmarks announcement?

I know I almost signed up but i'm still tossing it up between pinboard and historius
Most programmers in India live in their parents house. So they save $500 on rent and are able to live on $400 per month.

Programmers who move of parents house, share a 1 bed room apartment with 3 friends to save on rent and cover up their cost of living.

Few of them make over $1000 per month and live independently, with a better lifestyle. You actually need at least $1000 per month for decent living in India.

The $500 apartment I mentioned, is nothing fancy. Just a 1 bed room unfurnished apartment with bare minimum utilities and 12 hrs of daily electricity power cuts.
I can only imagine what it would be like to be inside a drill hole compressed inside a capsule and surrounded by rock for 20 minutes with the possibility of it being jammed. I can't help but to drag a comparison between surfacing out of this cramped chamber and birth. Instead of only the mother I'm sure the miners and their families will be under the effects of an endorphin rush.
Before running this test, did you estimate the sample size needed to detect the estimated difference in effects from A vs B? Since the expected difference is 0, the necessary sample size would be infinite. Therefore, we could at least estimate a very small effect, which would necessitate a very large sample. I would be very cautious, then, about trying to interpret the "confidence interval" before you had accumulated that large sample size. If you're going to peek before getting to that point, then you should almost certainly have implemented an alpha spending function (or similar) in order to maintain the desired overall error rate.

This should not reduce your belief in confidence intervals; this is a great, motivating opportunity that should prompt you to learn how to use them correctly.
then let it beat it. If its so much beater, it should be able to do without any special help.

we didn't have Steve Jobs going to DC to explain how his product is better than other phones and therefore needs a subsidy.

new products should prove themselves as they always have. it should not be whichever product that has presented for it the most widely believed theory that wins

if its so great, it should be able to garner investment dollars all on its own.
I also wondered how many of those millionaire educators are widows of men who had more lucrative professions.
Quicklisp public beta
Thanks for that, that's something I haven't seen anywhere. Do you have any references that explain why this error occurs?
I think that's one influence, but a great many people do it to impress their friends. You don't want to be the couple that had that bland, short wedding where there were no hook-ups, the dancefloor didn't go nuts, that one person didn't embarrass themselves, no one kicked on afterwards until 4am and people still weren't talking about it years later, do you?
No, this was about a month ago. You can give historious a go, as the account is free (as opposed to pinboard), so it's no-risk!
I'm saying that the mistake was overinterpreting the statistics without having a good handle on what they actually meant (part of option 'c' above). This comes off as ad hominem, but it's the most common error mode so I'd use this as a motivation to learn more stats and understand why the test was probably not telling you what you thought it was.
80% of arguments against lisp are superficial

And? Death of a thousand paper cuts is still death.
There aren't any for loops! I GIVE UP
This is one of the more insightful comments I've read lately. I thought just upvoting wasn't enough for this one, so I wanted to say thanks and I'm going to refine my next split tests along your guidelines.
I hate politics. They muck up everything and make it hard for me to learn the truth.

Right now, I am inclined to be suspicious of global warming folks, especially the extreme apocalyptic alarmist version.

I am apt to believe that global warming will be good for some people and bad for some people. However, when it is uniformity bad. Something is wrong.

I also dislike oil interests getting energy subsidy, or refinery cannot be built, and stuff like that. I knew that the higher price of oil will drive technological research for energy efficient.

I also thought the current pollution and emission crisis? have much to do with the government building roads all over the place, encouraging automotive usage instead of letting transportation evolved naturally.

Something is wrong with this debate. Especially if people are calling others unscientific for adherence to one side of the debate. This is not the bible contradicting the words of evolution, people.
In case it hits anyone else with a "login required", here's an account to use:

Username: getjiggly@sogetthis.com
Password: bugmenot

[Edit] Actually.. the log in worked for me, but it didn't give me the one-page version :/
No, we hadn't estimated the necessary sample size. To be honest, nobody expected the confidence metric to go so high for a test with no changes, which is why we were surprised. We've studied various A/B testing resources and we haven't seen this "peeking error" mentioned anywhere, so we'd appreciate any resources you may have (and would possibly summarise them in a subsequent post).

Thanks for the help!
When you take multiple peeks at the data as it begins to accumulate, you're essentially performing multiple testing.

Imagine that you wanted to see if A != B, with a 95% confidence. To oversimplify, this means that you're willing to accept that 1 in 20 times, you'll incorrectly reject the null and you will consider A different from B even though they are truly the same.

If you run 20 independent tests at once, each at a 95% confidence, then by chance you'd expect 1 to reject the null even if they're actually all null.

Now, if you repeatedly peek at the data as it accumulates for one test, you're doing something similar (not 100% the same because the tests aren't totally independent, but similar). To again oversimplify, you'd expect, by chance, to see "significant" results 1 in 20 of the times that you look at the data, even if there is no significant result. This is why you need to wait until you've collected all of the necessary data first, or implement a procedure to protect you from incorrectly seeing "significance" when it's not there. You might, for example, require a more stringent confidence level earlier on, and less stringent ones later.
Technology = Salvation: An Interview with Peter Thiel
Most people are capable of behaving inconsistently very true..

Its just that, I doubt it is limited to just religion and politics... When dug deeper and when looked across a person's behavior across different aspects, most people seem very inconsistent.

true consistency in idea/ideal and action...hmm...I would say that is an incredibly tough thing to achieve and probably impractical from many perspectives.
Another HP laptop Battery Explosion, user burned
Possibly this is some kind of political statement to show people how the internet looks in china?
Hmm, yes, I see now why you're right. I'd like to read a bit more on the theory, though, so we don't make these mistakes again. I'll look around for some resources, thanks again!
Sure thing. I just posted a bit more detail about this elsewhere in this thread. Also, googling "alpha spending function" brings back some useful results. (I can't paste into the HN text box with the latest Chrome dev, otherwise I'd paste some links here for you.)
The first person down is going to be a paramedic from the surface. It would be natural for this person to be the last one up.
Most bloggers encourage readers to contact them to submit anything newsworthy.
Terms to get you started are "power analysis" and "sample size estimation."
And this, ladies and gentlemen, is why we use Bayesian likelihood ratios. You can use whatever stopping algorithm you like on likelihood ratios and they will still be valid.

"Statistical significance", not so much. A simple test program I wrote showed that with a limit of 500 coinflips, flipping a fair coin will produce, on at least one out of the steps, a dataset which rejects the null hypothesis with p < .05, fully 30% of the time.

http://lesswrong.com/lw/1gc/frequentist_statistics_are_frequ...
I find it both antithetical and complementary. I can take apart my car to learn how it works, but my newfound understanding does not mean that the car was not assembled in the first place.

I once was reading a book on the history of science that was published shortly after Darwin's books on evolution came out. This history book discussed how many people thought that Darwin's ideas were meant to do away with God. This, the author claimed, was not the case: just because we have better understanding of how the world works does not mean that it is any less or more the result of a purposeful creation. It simply means that we understand more about how it functions.
Spain? The "S" in PIIGS?

http://en.wikipedia.org/wiki/PIGS_%28economics%29
I find the p.s. completely incongruent with the rest of the post.
Except there's no startup ecosystem and the culture is very traditional.
It can be a useful application.

Most startups will be happy to pay like $10 for access, if it can save them a few hours.
Yep, that's what I realised from your and jfarmer's comments. It does make sense that you're sampling multiple times and peeking to see if you've reached that, thank you.
The download part was no longer than 30 min. I am not sure how long the entire upgrade took because I left my computer and the upgrade stopped at one part because it needed to ask me a question.
Not sure if you are still a student or working full time.

I have few ideas but I guess only student would be interested to work on it.

These ideas wont make you millionaire but you will definitely make enough money to pay your rent and will help you learn how to do business in general.
Playing for a living is nothing like playing for fun. The psychological pressure to stay on form, the fatigue of playing hour after hour of poker, the uncertainty of chasing fish, the knowledge that your only career prospects are to play for higher stakes - it can be absolutely crushing.

The great paradox of poker is that the more you care about the money, the less you are able to think rationally and impartially. There's a strong argument that the only real difference between world class tournament players and averagely profitable semi-pros is the ability to keep playing your 'A' game when there are millions of dollars on the table. Plenty of skilled pros end up blowing their money because it just doesn't mean anything to them - the obvious example is Stu Ungar, who earned millions at poker and lost millions betting on golf.

It doesn't matter one jot how well you play at your best, what matters is how well you play when you're stressed and upset and ill and exhausted and it's 2am and the fish whose money you need to take is calling your mother a whore. That's the real job of playing professional poker - you're on a tightrope where an hour of perfect poker earns you $80 but an hour on tilt can lose you $800. The skills that allow you to turn pro are learned as an amateur, but there's a whole other set of skills required to stick it out without losing your mind. I had the former, but not the latter. I walked away from poker because I am absolutely certain it would have killed me.
If not bureaucratic fluff, what is to blame for the great success of organisations such as SpaceX while Nasa has done less with more?

Still, I concede that we must be cautious about potential silver bullets. Good point on the "libertarian utopian cure-all"
Right, but even frequentist methods can prevent an erroneous rejection of the null, if used correctly.
I saw that as well, thanks. This isn't the sort of statistics we did in uni, so it's always good to get extra pointers. Thanks again!