skysight-inc/HackerNews huggingface dataset:
Yeah, except both of them are modeling how people will vote, and people change their minds.

Saying that one should’ve held a constant chance of a candidate winning no matter what the polls do is frankly insane. This stance basically requires that you pretend that the outcome is determined and that voters opinions either don’t matter or don’t change. It’s utter nonsense, especially since we know that in 2016 undecided voters broke late for Trump. If your model doesn’t react to that, what are you even modeling?
> Who says the monolith we had was a clean codebase with a solid toolchain?

You didn't. They're just trying to force the issue for some reason by straw manning your decision.
I frankly don't care if someone is smug or not; I want to know who builds the best models. If Silver's models are often wrong, who would you recommend we look into instead?
Nice summary of the Nassim-Nate debate. A formal analysis/discussion of election forecasting is in this paper: http://www.stat.columbia.edu/~gelman/research/published/jdm2...

Edward says that Nate's prediction should be interpreted as:
“If nothing else changes between now and the election, Joe Biden has a 85% chance of winning.” (Silver’s argument)
But the problem is this prediction is not testable as election only occurs once at the end. Thus, one should use Nate's early predictions as pure entertainment.

Nate's final predictions show that they are well-calibrated across all of his political predictions but it is hard to estimate how accurate his model predictions are just for presidential elections given he has predicted only a few so far (unless one assumes all elections have similar uncertainty, which is clearly not true).
Yeah, this is the ambiguity GNU/FSF people go on and on about. “Free” in the interesting sense means “available to modify and use as you like, as long as you don’t prevent other people from doing the same”
Yeah, they missed what makes Emacs so customizable: its architecture.

What makes Emacs great is that the low-level text editor features are implemented in a compiled language and then exposed to an interpreted language to be composed into the actual editor.
If they're not splitting the architecture into a compiled part and an interpreted part, they aren't taking anything from Emacs.
I've recently come to believe that Taleb really doesn't understand probability. I know this sounds like some insane flippant remark, but I saw a video where he talked about the election forecast as a martingale bounded between 0 and 1 and then made some claims about variance with these bounds to back up his claim against Silver.

For his claim to make sense you would have to view probabilities as linear, which is not simply not how probabilities work. An event that has a 0.01 chance of happening is an order of magnitude more likely than one that has as 0.001 chance. However the different between and 0.5 and 0.501 is essentially negligible (hence probability is not linear because 0.001 does not mean the same everywhere).

I've generally given Taleb a pass as a bit of a crank that doesn't care about the philosophical interpretation of probability, but after seeing him misunderstand probability in such a major way I've started writing him off entirely.

edit: I should add I'm not particularly more convinced that
Silver really understands all that much either, just that this is all two very loud people that don't know much. Neither should be taken seriously. This isn't Bradly Efron arguing with Andrew Gelman.
Edit: as pointed out in the replies, I was wrong that the model was "explicitly" a now cast, and I would like future readers of my comment to know that without deleting things. I would still like to note that there was clearly a change in the 2020 model to increase variance as a function of time to the election when compared to the 2016 model. I think this is substantively the same.

Some of the top comments are making the same mistake, probably because they aren't aware of this twitter beef history (and I envy them), so I'm putting this in a separate comment.

These comments are all pointing out that Taleb's criticism is of something that Silver's models don't do:

> the election were to happen today, what is the probability of each candidate winning?

What people seem to be forgetting, is that Silver's 2016 model explicitly did this. Taleb made his criticisms starting in 2018 (even published an options pricing paper on the subject) then Silver changed his model for the 2020 cycle so that it no longer was an "if the election were today" model.

I can't say for sure whether or not Silver changed his model because of Taleb, buuuuut he deserves at least a little credit here. Yes, I know Taleb is a colossal asshole people love to hate, but he's usually right about these sorts of things.

Here is a pretty fair and balanced take on it for those who would like to read more: https://towardsdatascience.com/why-you-should-care-about-the...
eBay is the worst for so many reasons. I can’t believe no startup or even big tech company has tried to make a competing platform.
This was a correct description of one of 538’s models, the nowcast. It was never the one they recommended, and they did not bring it back later because it tended to confuse people. They actually made it very clear on their podcasts that the nowcast was a mistake.
I've bought a lot of useful things through instagram ads. TBH their targeting is really good and I in-fact do like the ads because they're so relevant.
Location: Burlington, VT

Remote: Yes

Willing to Relocate: dependent

Technologies: Javascript(React, Svelte, Vue, Express), Python, SQL/MongoDB, Rust, Digital Ocean

Resume: https://www.linkedin.com/in/robertstech/

Github: https://github.com/MichaelARoberts

Website: https://roberts.technology/

Email: michael@roberts.technology

About:
Studied Digital Forensics in school, went on to be a full-stack developer who has worked with numerous education technology startups as well as eventually working with Encyclopedia Britannica. I enjoy working with small teams and local brands. My current paying project is developing a custom shipping tool for a drop ship company. I have about 4 years of full-time development experience.
Is anyone else incredibly put off by the title of this article? It may be a pun, but an extremely insensitive one that just comes off as callous, especially given that we're in a period of time where very real repercussions of a careless attitude towards cutting-edge technology are increasingly coming to light (e.g. DeepFake being used maliciously).

The article's fine; seeing this title on my news feed just made me feel sad.
Well, that’s kind of on him to create, no?

Put another way, if trump wins...again, do you take that as part of the expected outcomes or evidence your distribution is wrong?

If he said something was a 10% likelihood, and it happened three times in a row, would you still think the prior is that the event was actually 10%, or that the estimate of a 10% likelihood was off?
I came here to say exactly the same thing. Silver and 538 are explicit about not doing this. Their models do include the uncertainty of future events, and do adjust closer to 50/50 given the same polls the further from the election it is. That's why Biden's odds have been ticking upwards slightly over the past couple weeks despite polls remaining pretty stable.

So in theory, the forecast should be exactly what it appears: the percentage of the time the candidate would be expected to win given the situation they are in at the time of the forecast. So if on Oct. 14 it gives Biden a 72% chance, it means that a candidate in Biden's position, with his polling numbers, and the economic conditions and everything else that is the case on Oct. 14, would go on to win from there 72% of the time.

I agree that it appears the volatility of the 2016 forecast was larger than you would expect given a correct model behaving that way. I expect that uncertainty is better expressed in the 2020 model. However it's certainly conceivable for such a model to swing correctly. IE, there's nothing impossible about a candidate having a 90% chance of winning on X date, then having circumstances change such that they only have a 50% chance at Y date. (In fact, if you ignore the discontinuity on election day, at least 10% of the time their odds would have to pass through 50% at some point, as they would go on to lose. And of course there would also be scenarios when it would dip to that level or below and yet still win.)
That's an interest take on it

I'd say the way of presenting results as a "sum to 100%" fails to present uncertainty in an intuitive way
As an American, I'd never heard of American Chip Spice, so I had to look it up (http://americanchipspice.co.uk/about/).

> "Where did this ground breaking product come from?" I hear you cry. Well (obviously!) American Chip Spice started in Yorkshire.

I don't have anything to add. I'm still laughing.
Same. Go back to rural midwest to visit relatives a couple times a year - I love it but after a week I'm reminded why I don't live there.
I think Taleb’s point is a bit more damning — implying the software does not do what it says it would.

If all sorts of things can happen, where you predictions change widely, Taleb’s argument is that you can’t use a number like 90%. You have to include the error, which ends up saying “50%” for both before the election.

This doesn’t mean that all prediction suffers from this effect, and you have to be skeptical everywhere. I don’t think that’s a fair characterization. Taleb has won by mispriced options. His main statement is that the true probability of this particular event is much closer to 50%
Every time a New Yorker article shows up, people crawl out of the woodwork to complain that it's a New Yorker article.

If New Yorker articles are not for you, then do not try to read New Yorker articles. Really! Read other things you will actually like. If you want a quick rundown of the topic, Wikipedia is only a click away, and gets right to the point.

Those of us who like New Yorker articles appreciate them when they appear. We know there will be an investment of time, and we know that a board of Editors has ensured that, for those of us who like the format, our time will not be wasted. Those who don't, know in advance that they will just be frustrated and bored, and can skip past instead. You are not even obliged to "upvote" the post.

Really, nobody is keeping track. Nobody will think ill of you for skipping past the moment you notice "New Yorker" in the URL. If you ever become ready for New Yorker articles, New Yorker articles will be there for you.
Completely unmentioned is reliability of infrastructure and scope of integrated systems for OVH vs AWS. OVH looks good on paper but wait until there’s a problem. Downtime cost is usually far more costly than just server cost.
This isn't true. He had both a "nowcast" (if the election were held today) and a prediction of the final result. The only change in that regard between 2016 and now was to stop publishing the nowcast.

Edit: Even after your edit I don't think the claim is true. The probabilities in this election didn't change much because the campaign season was comparatively uneventful and the polling didn't change much, not because of a fundamental change in the model. 2016 had tons of bombshell events which is what caused the chaos in the prediction model.
> Whose models are better? There have been years where Nate Silver's model has called every single one of the 50 states correct as far as which POTUS candidate they'd vote for.

Well, this certainly isn't how you'd go about evaluating model quality. Calling which candidate a state will vote for is not a good task; it's too discrete to be a good evaluator. For the same reason, no political model produces this output. You predict the vote share given to each candidate -- a nice continuous variable -- you don't predict who will win. Calling the state is a gimmick on top of that (since victory is a function of vote share, it's easy to degrade your model's output into victory predictions) which serves only to make the statistics worse.
One of 538’s models did that in 2016, not all of them. They made it very clear that they did not recommend the “nowcast” because it tended to confuse people.
Eh... what turns me off is that list of features and other editors that just names off other editors as inspiration. Don't even mention other editors unless you are fully copying their features.

Imagine a car being advertised in a similar style called Imacar.

Ferrari, know for its Italian design and sleek lines, Imacar takes those famous design cues by including wheels and tires.

Toyota, known for it's rugged truck's like the Hilux. Inspired by that ruggedness and repairability the Imacar includes easy change windshield wipers.

Tesla, know for their famous electric motors and batteries, inspired us to add a battery to Imacar's petrofuled engine.
The problem is there is no middle ground. FP people forget about the product because of FP. Alot of non-FP people forget also forget about their product because their are swamped with unecessary maintanence. There is a middle road in there somewhere where you use just enough FP to avoid the problems you would face without it, without getting totally sucked into making it perfectly consistent.
Why don’t we use blockchain for accurate, anonymous polling? Rewards in BTC?
Google’s reCAPTCHA service: Questions raised over privacy promises
That was a very well-written comment, thank you. Hit the dopamine and gave me a smile for your explanation.
They do do that analysis, in some detail: https://projects.fivethirtyeight.com/checking-our-work/
Mostly due to interest in FP being driven by intrinsic desire to use it, since it's so rare in enterprise. Anyway, a lot of that discussion filters down into more popular languages, so it's pretty incorrect to think of it as all academic junk or something.
There were models like that, they didn’t bring them back this year.
The prediction should have more data to it about variance, but it isn't necessarily wrong.

We have good data about rain, I am sure we could find out what percentage of the time it rains in New York on a given day during that time of the year. There might be high variance, but that doesn't mean it is wrong to say there is a 2% chance it rains on a particular Monday.

This is how insurance works, how sports betting works, how option pricing works. Just because something has high variance doesn't mean we can't give probabilities that work out over a large number of samples.
You're a funny guy. Nothing is wrong with a business making money. What's wrong with Amazon is that they're going to make extra money through extortionate business practices and you're going to be shit out of luck looking for an alternative after they crush all of their competitors in ways that are not healthy to a functioning market.
Lambda | Senior Software Engineer | Distributed | Remote | Full Time

Remote first! We're looking for a software engineer who wants to write code at our Deep Learning workstation and server company.

You will:

* Be responsible for the full stack

* Build reliable, scalable, highly available systems
Required Experience:

* Bachelor's degree in technical field (CS, Physics, EE, Math, etc.)

* Extremely comfortable in a Linux/Unix shell

===

Lambda is a distributed remote first company.

Apply to:
https://lambdalabs.com/careers
The irony of this is that the RPi is far more closed and proprietary than regular PCs and their components, of which you can find plenty of listings for on eBay...

I hope this was just a mistake and done due to other reasons (they thought these were fake RPis?)
It was not. You are mistaken about the history.
That's incorrect. At _least_ as far back as 2016 and I believe quite a bit farther back the model was not "what if the election were today?" There even used to be a separate side-model that was explicitly that, the "nowcast" I think it was called.

I suspect that 538 _never_ covered an election solely or even primarily with a "what if the election were today?" model, but my memory isn't that reliable back that far.
Taleb’s point is not that your prediction should not change.

His point is that if it does change, and frequently at that, the purported 90% chance at t(n) is bogus. That prediction needed to account for the volatility
SEEKING WORK - Remote - San Francisco, Tokyo, Singapore

Need an MVP in a month? Here's a recent example:

https://tako.social

-> Senior full-stack software engineer
-> Domain expertise in advertising, e-commerce, and fintech
-> 10+ years of experience at both brand name tech companies and cutting edge startups.

I can build your vision from scratch or turbocharge your existing product development. No project too big or small.

Examples of prior work include e-commerce checkout flows, social network startup prototypes, microservices and APIs, financial reporting, Intercom-style storefront Javascript widgets, behavioral tracking, analytics dashboards, internal tools, and refactoring existing legacy applications.

Proficient with Rails, React, Angular, Typescript, SCSS, Bootstrap, MySQL, Docker, AWS, Google Cloud.

Recent feedback:

- "Indispensable"
- "Awesome colleague... always ready to help ... positive attitude towards work is contagious ... a true team player"

Web: https://tholf.org

Contact: hn at tholf.org

GitHub: https://github.com/tomholford

LinkedIn: https://www.linkedin.com/in/tom-holford
Now you've lurched from denial to some sarcastic realist approach where we're shamed for suggesting it is even appropriate to be concerned about the issue.

Please consider reviewing the posting guidelines
I think you can either:

* make predictions with a range
* or make predictions with a certainty coefficient (p value, etc.)

Predicting a single value without either is not a truly quantified "prediction."

It's weird now not getting that can lead to confusion. I agree with others in that those who don't communicate both values of information aren't being 100% clear.
You seem to be all over the place with your issues. Are you mad that AWS has bandwidth limits, or are you mad that AWS is encouraging the use of Docker altogether, or do you just have some kind of vendetta against the boogeyman that is AWS?

Docker registries are how things are done right now. And the changes to Docker Hub that happened this week have a significant impact on the Docker ecosystem. This announcement is about solving that impact now, because developers need a solution now. This doesn't have anything to do with any hypothetical improvements to the Docker ecosystem or Docker data usage that may otherwise be in progress by [insert company here].
> one problem I've always had with Silver is that he's very smug about this models but they often enough don't work out.

What does it mean for a probabilistic prediction to "not work out"? If I tell you that the odds of flipping 2 heads in a row with a fair coin is only 25%, and it happens, did my model "not work out"?

> instead of reflecting on his models to improve them (at least publicly,) he just tells people they don't understand probability or his models.

I really don't intend this in any sort of rude way, but I think you might be interpreting his response as dismissive because it's accurate...

It's difficult to say a model "doesn't work out" based on a single event, especially when it puts a significant probability on the less-likely option. For example, 538's 2016 forecast gave Trump a roughly 1 in 3 chance of winning: that's _higher_ than the odds of flipping 2 heads in a row.

Measuring the quality of a model is more complicated: one way would be applying the model to the same outcome multiple times, but this isn't possible for single-event forecasts. Another way to see how calibrated your predictions are is to aggregate multiple predictions and see how often they line up with reality: an event predicted to occur with X% probability should occur X% of the time.

Lucky for us, 538 has done exactly this analysis[1]! Naturally, it's internal, so take it with a grain of salt, but it looks like their predictions are fairly well-calibrated.

[1] https://projects.fivethirtyeight.com/checking-our-work/
My favorite story when I hear someone dismiss an argument saying 'that is obvious':

Lazarsfeld was writing about “The American Soldier”, a recently published study of over 600,000 servicemen, conducted by the research branch of the war department during and immediately after the second world war. To make his point, Lazarsfeld listed six findings that he claimed were representative of the report. Take number two: '“Men from rural backgrounds were usually in better spirits during their Army life than soldiers from city backgrounds.”

“Aha,” says Lazarsfeld’s imagined reader, “that makes perfect sense. Rural men in the 1940s were accustomed to harsher living standards and more physical labour than city men, so naturally they had an easier time adjusting. Why did we need such a vast and expensive study to tell me what I already knew?” Why indeed.

But Lazarsfeld then reveals the truth: all six of the “findings” were in fact the exact opposite of what the study found. It was city men, not rural men, who were happier during their army life. Of course, had the reader been told the real answers in the first place, they could just as easily have reconciled them with other things they already thought they knew: “City men are more used to working in crowded conditions and in corporations, with chains of command, strict standards of clothing, etiquette, and so on. That’s obvious!” But this is exactly the point Lazarsfeld was making. When every answer and its opposite appears equally obvious then, as he put it, “something is wrong with the entire argument of ‘obviousness'”

More here: https://www.newscientist.com/article/mg21128210-100-the-huma...
Treat artificial light like others forms of pollution, say scientists
Maybe not Saigon, but as a night owl, it's been a dream of mine to be able to slide my 9-5 EST jobs to being 2nd shift jobs, like say in Central or Eastern Europe.

People have different biological clocks, and being able to detach from geography and perfectly tailor your clock to the company's clock, frankly, sounds like good efficiency to me.
To be perfectly honest, I think the Obama elections were a unique point in politics where models like Nate's worked a lot better. Realistically there weren't many up for grabs states, which means you could guess 8-12 states right and run the table. A lot has a evolved since then, especially in the social media landscape and the accelerated death of answering phones, which live polls rely so heavily on. Maybe Silver will hit this election out of the park, I don't know, but I would err on the side that he gets a number of states wrong.
That's what confidence intervals are for. You cannot condense both your expectation value and uncertainty in a single percentage (it is not correct to say that the probability of an outcome is 50% if 80% of likely outcomes result in one candidate winning). Otherwise you'll end up with every 5-week weather forecast saying there's a 50% chance of rain.
I imagine an auction-site like this is only as good as the number of people that are using it.

What issues do you have with it? I find it's not perfect, but it generally works reasonably well.
Looking for: Contract jobs

Location: Oslo, Norway

Remote: Yes

Willing to relocate: No

Technologies: C, digital signal processing, electrical engineering, embedded, ESnext (vanilla, vue.js, node.js), hybrid circuits, Java, multimedia programming, OpenGL, PCB design, POSIX, Python, SoC

Email: thj@thj.no
Agree; from my experience with Pi SD cards, it’s straight-up irresponsible to recommend that anyone use an SD card with the Pi for their main computer.

Every Pi SD card has ended in irreparable disk corruption. The SD hardware’s wear-leveling and delayed allocation is fundamentally incompatible with any device without a battery, especially with frequent writes.

The “official” solution for the Pi is to use read-only SD cards with overlayfs. That’s not the default and it’s not how entry-level users will use this.