skysight-inc/HackerNews huggingface dataset:
I'm not very familiar with Silver's background. I was about to write up how similar this type of analysis (and the ensuing reaction to it from the established pundit class) was to the baseball world and the rise of sabermetrics. Then I saw his background and found out that that's where he started. Not surprising.
If you wanted to put Microsoft under a microscope from 2003-2010, during the time where they were actually putting in the work to transition from a 1990's software security practice to a 201x security practice, you'd find plenty of "smoking guns" to win arguments with on message boards.
I wasn't aware of that. Thank you for reposting.

And yeah, snarkiness seems to be my involuntary hallmark... ;)
A gist.io of the article to ease the sudden load on the site (or if you simply can't get to it due to 50* error):

http://gist.io/4007765 (Source: https://gist.github.com/4007765)
Obama Crushing Romney in Bumper Sticker Poll
Can what Nate Silver be doing be considered science?

He's dealing with terribly sketchy data. Response rates to political polls in America are south of 10%, and there's no proof that the portion of the population willing to have a conversation about their political preferences with a stranger is representative of the population as a whole.

He's also dealing with incomplete data. The 'likely voter' screen pollsters use to determine who's actually going to the polls isn't always revealed, and it varies from organization to organization. Polling is a cash-strapped industry and some polls' likely-voter screens are much more porous, cutting down on staff required and voter contacts needed to generate a 'statistically-significant' sample. But details of the likely-voter screens and response rates aren't always available.

His results are also completely unverifiable. Let's say his final prediction is 80-20 in favor of one candidate, but the other candidate wins. Well, his model did say that'll happen one time in five, so you can't really criticize it.

The appropriate amount of precision to conclude with, given all of the fuzzy inputs, is something like 'well, we can't tell who's going to win - it's close, although this one candidate's chances look slightly better'. It's not 'Candidate A's chances improved from 74.6% yesterday to 76.5% today'.

That's what makes Nate Silver so irritating - he doesn't know any more than the journalists, but he claims he does.
I think he's referring to the BizSpark program: http://www.microsoft.com/bizspark/
Weird, in '08 he had a page giving the model in detail, including some of the fit values (e.g. the imputed pollster "house effects"), but I can't find it this time around. Change since the NYT purchase?
No, the saying does not hold. Microsoft didn't buy "many eyes"; they bought a relatively small number of very specialized eyes. At any one time in the mid 2000's, something like 4-5 security firms did $1MM or more in a year at Microsoft, and those firms each had between 15-40 people working at them --- and no firm did 100% of its business at MSFT.

What happened at Microsoft may not disprove this folk wisdom about defect detection, but it's evidence against it, not for it.
I have never seen this blog format - html uploaded and shared on Dropbox. What are the benefits? Anonymity?

It seems that many blog for publicity or driving traffic (both valid reasons), which leads to using simple publishing tools (tumblr, svtble, weebly, etc), custom domains, links to the author's other content, a personal profile, and an overall media-centric approach. Curious why the OP went with this approach!
Punditry is analysis of known facts.

Journalists do it through their prism, statisticians do it through theirs (and yes Nate Silver is a statistician and so his assumptions will affect his predictions)

Other journalists perform analysis on secret or private facts. The leaks, off the record briefings of normal political discourse, as well as actual real investigative journalism (both of them who are left)

Pundits who don't get the difference are doomed to be replaced by pundits with a copy of SASS.

Investigative journalism is just doomed.
I do not have a comment on whether an obligation exists.

I do however want to point out one personal benefit to pursuing a philanthropic career: your coworkers will tend to be more caring than non-philanthropic co-workers are.

Remember Joel Spolsky's short summary of what makes a good employee -- "smart and gets things done"? Well, "smart, gets things done and cares enough about other people to make helping them an important part of his or her career" is even better.
The file was modified Oct 21st, which was right before the iPad Mini was announced. As has already been pointed out it's on every other international site too.
>There are two possible answers for why this is. The first is just the sheer size of the big banks. These are large organizations and it's impossible to police everyone's conduct.

If large retail banks can figure out how to track and subsequently arrange a string of overdrafts to maximize "insufficient funds" fees for millions of personal banking customers, their investment arms can certainly track the transactions of thousands of employees.

>Now, there's a second way of looking at this problem. According to this story, senior bank managers are well aware of the risk of rogue trading, and have, in fact, set up the risk management systems in a way that makes at least some rogue trading expected and almost inevitable. The idea is they give their traders leeway.

As long as the bets are winning or not losing too bad, he's just a trader. When a big enough bet loses, he becomes a rogue.

The banks are complicit and these "rogue traders" are the fall guys.
Of course, US Govt is not the biggest of our problems. Others include:

- Rogue (or just bored) employees of web companies accessing customer data for fun or profit. (Happens much more often than you think, also in govt agencies.)

- Internet criminals breaking into cloud accounts and stealing data.

- Companies using their knowledge of their customers against those customers in disputes and legal challenges.

- Companies trying to extract the most financial value from customer data by selling it to questionable outfits.

- Foreign intelligence services and outright criminal organizations getting access (through 'hacking', bribery or threats) to any information hosted by any web service and many government institutions.

[I mean, for Christ sakes, if a news publication (NoTW and other tabloids) can buy some very private data of celebrities from UK police, how hard would it be for an organization with bigger resources and no fear of legal retribution to access any electronically stored data - especially by companies?]

* * *

And yet the trend in our merry startup world is to put everything in the cloud. Try asking any web company for a self-hosted version of their service.

For instance, can I get a Evernote server software to roll my own Evernote server? (Compiled and obfuscated, encapsulated in a VM appliance, I don't care.) Even if I was willing to pay for it like for any other software? No. Actually, Phil Libin was asked about it on the Triangulation show on TWiT. His answer? (paraphrasing) "Well, um.. It would be hard, um... The real question is: how do we make you trust us." Well, if that's your answer, you've already lost me. I mean, go ahead, keep my recipes and random silly photos. But if you expect me to trust you with my private documents or my schedule or anything of any IP value from my work, then you have a much bigger problem than communicating.

And so does your company, dear HN reader.

(Well, unless you're doing something frivolous, of course. If you're into the next FartingApp™ or photo-sharing-with-a-twist website, then I guess you're safe, for the most part.)
Nice slidedeck for good APIs principles evangelisation
But nothing on Policy and Terms of Use which become the main issue for developers , before good design.
I found out today that the US weather service gives a % chance of rain estimates. That's good but not enough.
The implication here is that Microsoft didn't actually improve their security so much as Oracle and Adobe failed to keep up with theirs. I don't know whether you intended to say that, but either way, it's a false statement.
If it is true arbitrage (that is, no price risk), it is very rarely in the same market.

e.g, let's say you can buy 1 EUR by paying 1.2 USD, buy 1 GBP by paying 1 EUR and buy 1.3 USD by paying 1 GBP, you would be left with 0.1 USD with no risk - in this case you've made an arbitrage profit of 10 cents, and you're likely to be able to do that "in the same market", and won't have to do more than one thing simultaneously.

However, everyone is looking out for these, so if such an opportunity presents itself, everyone tries to take advantage of it, thereby changing the price; These opportunities last milliseconds or even microseconds these days, the profit that can be extracted is very small, and you need to be very well positioned (technically) to be able to make it.

Another form of (mostly true) arbitrage is when the same thing gets traded in multiple venues - e.g. Gold in NY, London and Asian markets. In this case, you CAN'T buy a bar in one place and sell in the other place at the same time, because you need time&money to transfer the metal. So you go long in one place, go short in the other place, and then wait for the prices to converge WITHOUT trying to move physical gold around, and do the reverse transactions. There is no real price risk here (if the prices never converge, you CAN move the gold bar around for a small cost), but there's a lot of procedural and counter party risk.

There's what's known as "stat arb" (statistical arbitrage), which is all statistics and no arbitrage; Let's say Gold and Silver tend to both move together (when Gold moves up or down by 1%, silver tends to move in the same direction by 1%). Now, you see Gold moved up 2%, but silver hasn't. You assume either gold will move down 2% back to silver, or silver will move up 2% to match gold, or gold will move down 1% and silver up 1%. The way to take advantage of that is to sell gold for an amount $X, buy $X of silver, and wait. Whatever scenario happens, your balance (when you sell your silver and buy back the gold), you will make money. Unless ... gold and silver stay divergent, which can happen. In which case, you'll lose money.

Thanks to financial wizardry, you can very often sell things you don't own, and "buy them back" later to make things whole again.

(And finally, there's something known as "risk arbitrage" - it is all risk, no arbitrage, but it sounds like you're not just gambling if you call it "risk arbitrage")
"The worst is the regression of html, xml and javascript we are forced to use.."

History repeats itself.

Full screen apps, to me are a throw back to the 80's (before Windows and the Mac); and "Cloud Services" sounds awfully like time sharing to me; Centralized access point for distributed data reminds me of running apps on the old mainframes; Time Machine - real time TMS (backing up to tape in real time) - the list goes on and on.

All the kids thinking they invented new ways to share and use data. I feel like I'm knocking on a door reminding them that none of this is new. It's just been re-branded.
I am the OP, and I don't normally blog. This format seemed to be the simplest solution.
Signed up! I don't use Ruby much, but I'm curious to see what I can steal from Ruby back to Perl. :)

BTW, you should include links to the other 3 lists on all the websites. I see that statuscode.org and rubyweekly.com have it now, but when I checked the JS site earlier I couldn't find a link.
IPad Mini: "Time to kill pixel as the gold standard of browsers. Let's use cm".
Ask HN: my IP seems to be banned on HN, now what?
It seems my IP was banned by Hacker News somehow. I haven't done anything malicious or posted anything that would get me banned. I am however working on a small chrome extension for HN and I did refresh HN quite often. Other than that I had the "Hacker face" (https://chrome.google.com/webstore/detail/hackerface/daljejehnbbbhjlecjgafnnfgilbkdhj) extension installed, which I've now uninstalled. Other than that I don't have any extension installed that would trigger automated requests to HN (my extension makes no requests).

What can I do? I'd really like go get back on HN :)

ps: I'm able to use my account and browse HN if I VPN into my office ...

pps: my IP is: 89 212 118 68

edit: I get this when I try to open the page in Chrome: Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data.
I think the premise--or at least the wording--is wrong. Journalists like Nate Silver. It is political pundits (who are almost the opposite of journalists) who can't stand him.

Fittingly, pundits typically dislike actual journalists for the same reason: they attempt to report things accurately rather than through a party or ideological filter.
This is a very polite interpretation of (some) traditional journalists' dislike for Mr Silver. It offers a consistent explanation for the dislike and vitriol, but there is another explanation to explore. Silver presents a method of investigation that is not only epistemologically different from traditional punditry, but threatening to it, and their dislike is centered on the threat his method poses, not just its intellectual merits or misunderstanding thereof. There are plenty of things for journalists to intellectually object to - why focus their efforts here?

Consider that Mr Silver has been given awards and accolades most journalists would kill for: Time's list of 100 most influential people in the world, best political blog Webby award, Rolling Stone's 100 agents of change, the editor in chief of Politico listed him as "one of the most powerful people on earth", he's given a prestigious lecture at Columbia Journalism School, etc. He's received accolades, even from traditional journalism sources, that most journalists, even at the NYTimes, would never dream of. Speaking of which, the Public Editor of the Times wrote "he’s probably (and please know that I use the p-word loosely) its most high-profile writer at this particular moment."

All this for someone who has never worked a proper newsroom since college, never had a television show, cable or otherwise, and just started a blog four years ago because he was (supposedly) annoyed at laws that threatened his livelihood - online poker.

So imagine seeing someone like this receiving all the awards you've coveted since you started journalism in earnest. While the recoil against Mr Silver might be clothed in intellectual differences for some, the reason they care probably has more to do with seeing someone who differs from their traditional approach getting so much recognition.

Finally, I'd note that there are many traditional journalists who support Mr Silver's approach - his awards from traditional journalist organizations speak to this. Rather, it's a very specific group that seems to feel threatened.
I am not sure if the app is using the right dictionary or if it is broken in Chrome. I used the string "AARGHQQQQQQQQQQQQQQQQQQQQ" to run a search and it returned no results. "aargh" is in the Letterpress dictionary.

I wrote a quick Python script last week that took the board position and returned the unplayed word with the highest point values (with a winning move being worth the max value). I thought about expanding it to look ahead, but it wasn't needed to beat human opponents.

The coup de grace would be to bundle it all up as an app that pulled from the camera roll and spat out a move.
Cinnamon is great. I'm actually using a gnome classic DE right now, but i suspect that I'll switch to cinnamon when it becomes more polished.
Mark Cuban Responds To His High Number Of Fake Twitter Followers
I'm not sure the Godwin was necessary or appreciated, but would you mind elaborating a bit?
political polling data is not independently testable
>His results are also completely unverifiable. Let's say his final prediction is 80-20 in favor of one candidate, but the other candidate wins. Well, his model did say that'll happen one time in five, so you can't really criticize it.

That's a good point. He is using percentage chances everywhere but then running only one trial (election).
Thanks for this. One of my goals is to get my fellow students more involved in the building side of things and not just the academics. Get some real world stuff going.
Okay, but let's say you want to test a few looks. You tipically change back and forth and make small changes, so a few seconds for the redeploy, and mabe a few clicks here and there to get to where you want to be. That might take 10s each time. So testing if what looks nicer is is a matter of instant change and 10s per change and extra clicking all the time. I personally don't like repeating anything more than 5 times if it's not necesery.

If the state that you're in is more coplicated, you might need to type something, log in and do some crazy stuff to get to where you're debuging, then you see how much more anoying that gets.
I wanted Twitter so I clicked on the "t." but that's actually for Tumblr. Looks cool, but I'm not sure it accomplishes its goal.

I would think it would be much more useful to write out "Twitter - This is the site that lets you post short messages or read short messages posted by friends and celebrities"
Why is it unreasonable to expect the customer to buy their own hard drive and back up their own data?
Show HN: Using Facebook Statuses as a CDN
A brief guide to computational thinking
The other big advantage on mobile is a closed ecosystem and mandatory sandboxing.
This seems like a lot of unsafe work just to avoid a temporary copy of the string's byte encoding. Isn't the garbage collector tuned for exactly that sort of short-lived data?
Yet you are viewing this page a with browser that loads and automatically executes untrusted JavaScript code under managed runtime.
If there's a salary difference, take the higher paying job, and donate the excess to a foundation who employs people to work on the humanitarian issue you're most passionate about.

That way, society gets the benefit of both your skills and your passion.
I don't think Silver claims to have data or knowledge that is better than what is available to the public.

What he does different - and his isn't the only model built off of polling data, nor are 538's probabilities that different than other models - is just trying to decipher all the available polling data to reduce the noise and focus on the signal.

I also don't think he sees himself as being in the business of picking a winner, as in saying "I believe X will win tomorrow".

One thing you might be missing is that his predictions are verifiable at the state level - a result where he predicted the overall winner but only got 30 of 50 states right versus getting 49 out of 50 states right are pretty different.

Polling aggregation and models are as much "science" as the practice of polling is in the first place.
http://news.ycombinator.com/item?id=4722667
Ranker: Crowd Sourced Top Startup Blogs
Can someone please explain how the judgement has effect throughout the EU if other courts in the EU judge differently?
And for most of the 2000's, browser security was a debacle, too.
Silver tries to be transparent about his model. For a non- technical description, see [1]. He used to have something more technical/concrete on his personal blog. I'm guessing you can still find that easily.

The fact that he doesn't include specific equations in his NYTimes blog is likely only to improve accessibility to non-technical readers.

That said, he has given enough detail for others to roughly replicate his work. See [2]

[1] http://fivethirtyeight.blogs.nytimes.com/methodology/

[2] https://github.com/jseabold/538model
Key management is beyond the average user of the internet.

Maybe it'll improve if we start putting encryption keys in physical charms that you can hang on your real-life key-chain; then again, there is no locksmith, so maybe not.
I suspect every single person who becomes aware of the absurdity and lies (and who is not an insider) things that the collapse is a couple of years away. Since (at least) 1971.

I'm not saying the collapse is NOT near (things are getting worse at a quicker rate and are becoming harder to hide), but it is a well known adage is that "markets can stay irrational much longer than you can stay solvent".

They've been irrational for somewhere between 30 and 80 years. I suspect the irrationality will continue much longer than one can rationally expect.
Draftsight Review: Like AutoCAD, but free
"...but if anyone wants to buy my company for a few hundred million (it’s worth it!) then I promise I’ll follow in Elon Musk’s footsteps and focus on the rest of you."

I honestly don't know how to take this.
No, the judge cannot do anything at this point. The exact notice is on the page. So what if you have to scroll to see it.
> a product recall which is your fault in the first place

First: it's not a mandatory recall. You didn't need to replace the iMac hard drive.

Second: Western Digital hard drive failures are not directly Apple's fault. Apple package the product but it's only indirectly their fault -- Western Digital screwed up here.

Third: The affected products are all past their warranty date. Apple are being better-than-average by offering this recall at all.

Does this makes Apple look like a "dysfunctional, hideous and dystopian company"? Not in my opinion. In fact, you sound like you've gone completely off into ridiculous hyperbole-land.

> I am surely able to do it, considering the poor skill level of your staff. You should offer this as an option.

You don't have any backups of your hard drive (which is why the easy options all disappointed you) but you think you're skilled enough to replace an iMac hard drive? Do you have a full set of Torx screwdrivers and suction cups? Maybe you should sell your Torx screwdrivers and buy a backup drive.
Show HN: Make your own presidential predictions page.
I wanted to play around with raphael.js/CSS/PHP and put this together.
It lets a user make their state by state predictions, and creates a custom page that they can share with others:

http://pollster.net

I'm hoping to get some feedback on the good, the bad, and the ugly. I'm new at this, and hoping to learn something.
I think because this was part of a product recall. Apple says "hey, we screwed up, we need to replace your hard drive", so you'd think they'd provide the new one and do the back up themselves, instead of "we screwed up, you should drop $100 to fix the situation we put you in".
It also underscores the value and incredible importance of reliable infrastructure and a well-oiled economic machinery. We're all humans, we can all use our time and energy to do "useful" things for ourselves or our peers. But the amount of useful stuff you can get done during a day is extremely dependent on the available infrastructure and support machinery.

This is incredibly obvious today when you can rent as much world-class computing machinery as you want at a moment's notice just by using your credit card, and when you have libraries and languages that took hundreds of man-years to create available for free. But the same is true all the way down to the point where we are running around in the jungle, naked, picking berries and nuts and attempting not to get eaten by lions. Computers, electricity, plumbing, roads, debt, currency, money transfers, automated farming, artificial fertilizers, remote communications technology, materials technology, engineering and physics, chemistry, mathematics, written language and spoken language.

It's all a giant pyramid of human accomplishment that allows us to produce wealth/do useful and interesting things so much cheaper and simpler than before that it is simply ridiculous.
The 20 Most Innovative People In Democracy 2012