skysight-inc/HackerNews huggingface dataset:
Probably good to regard these first few weeks (months?) of Pokémadness as an "open beta" period, before the security measures get turned on. We can look at Niantic's previous project, Ingress, for a roadmap.

The two major categories of cheatifying in Ingress are falsifying one's location and multi-accounting. There's precious little that can be done about the latter, so Niantic focus on banning players that appear to be "spoofing" their location.

Given the wealth of different devices and playing scenarios, immediate detection of GPS spoofing is infeasible. Things like WiFi router locationing idiocy (or even just dodgy GPS antennae) play havoc with the utopian dream of perfect positioning every time. If a player performs actions seconds apart that are separated by thousands of miles then the game temporarily ignores them, but after some time in the naughty corner they can resume play.

Hardy spoofing detection instead depends on longer-term profiling. Ingress has a similar API to Pokémon Go – JSON chunks (rather than protobuf) over HTTPS, most fields out in the open – but each request from the app includes a monolithic "clientBlob" containing device characterisation. The format of this has been (presumably) reverse-engineered by a few hardy souls but it is certainly closely-protected Niantic knowledge. We could safely assume that it's a proprietary blend of signal strengths, gyroscope readings, touch events and timings, secret herbs and spices etc.

The clientBlobs lend themselves to offline processing. There are conceivably servers continuously trawling through a backlog looking for tell-tale patterns of bad behaviour, but it also provides an audit trail if a particular player is suspected of spoofing. Occasionally Niantic indulges in mass purges, which presumably follow from a new cheat detection heuristic being run on all the collected data for some period. These "ban waves" have a reputation for penalising unusual device configurations (the most recent major wave appeared to target, amongst other things, players with modified Android variants that might mask GPS falsifying code, including cheaper Chinese knock-offs, and Jolla phones running Sailboat).

Occasionally during major Ingress gaming events – so called "XM anomalies" – there is some level of human supervision to quickly identify and remedy clearly-fraudulent player behaviour, but for day-to-day operations it seems that account termination, so-called "hard bans" and shorter-lived "soft bans" are entirely automated, and based on offline player data analysis.

Getting back to the New Cruelty: the clientBlob was not part of Ingress's initial implementation; for a while after it was introduced was ignored, and then it became mandatory. A similar opaque chunk of data is included in the Pokémon Go requests, so we should look forward to its imminent deployment when Niantic scrape together enough Pokécoins to buy a few new servers for batch processing. At that time these convenient APIs won't have long to live.
I don't ever want another phone with CynagonMod again. It's awful.
Take a minute and reflect on the fact that you're arguing Turing had nothing to do with mathematical logic.

His doctoral advisor was Church, his work based on Gödel, his thesis named Systems of Logic Based on Ordinals.

Comments like these show the perils of treating mathematics like a cookbook. It's a very limiting mindset.
I'm not the author, but I'm a huge fan of robust regression. I make between $500-2000/month off a trading strategy based on such a method. (The method is basically Bayesian linear regression, but using an error model that has a heavier tail than a gaussian.)

But a really important thing when using such methods is the lucas critique. When you need to use robust regression you are definitively in a space where all the simple and generic stuff (e.g. linear regression) doesn't work. So at this point it's important to validate the underlying assumptions behind the robust regression scheme.

E.g., in my trading strategy, I've gone to great lengths to make sure the tail behavior I'm modelling is an overestimate of reality.
We have a job to live, we don't live for work.

I recently changed career to become whatever I found honest just in case it would make my life different.

Well, being a human beings seems to come with a handicap we almost all have moral compass. And most of my job in IT have been in the dark side of my compass recently.

Right now, I maybe risking my body moving heavy loads with poor equipment and security, commercial making occasional mistakes forces us to do 12h continuous loading of trucks else the company bankrupts and none of us are paid BUT from my perspective it is a great improvement.

Customers are sometimes saying thanks. Coworkers are sometimes saying thanks, and boss too. We are working as a team and when everybody does his/her job correctly we have satisfactions.

A satisfaction I was missing.

And when the day is over, the job is not in my head anymore.

I can once again live a normal life, we don't scam customers, we don't break their goods, we are the most honest we can giving the stupidity of some regulations and of some dishonest customers.

And fuck, being able to feel proud again is worthy the quasi state of misery I live in.

Sometimes, money does not matter as much as feeling you are not wasting your life doing something that makes you something you will come to despise.

Feeling an honest human again worth every single $ and all my savings I lost in the conversion.
I suspect they're planning 'CyanogenMod GO'.
Paul Manafort made a career out of  reinventing tyrants as defenders of freedom
Saying they are not successful in Smalltalk doesn't jive with the history of refactoring browsers in Smalltalk. I doubt the widespread use of a tool in a language if it didn't work.
Doesn't it? Not to challenge the authority, but I believe the only requirements OTR impose are constraints on the initial presence (thus, lack of offline operation) and constraints on message ordering. Which doesn't seem too severe to me, if the saying about "always connected" world is true enough in reality. Maybe I'm deeply mistaken here. But, I mean, 99% of my conversations are when both participants are online, and it does work for me in practice (and I can't realistically replace it with Signal) - that's why I've had the opinion that it's still haven't gave up on "gold" yet.

As for ubiquity - 2 billion devices is great (and hope there will be more!), but I meant the other sort of it. I've ran OTR sessions over XMPP, ICQ, Skype and Telegram. I guess, it's a wrong sort of ubiquity, probably not something that works for the goal of "encryption for everyone", but it still works if I don't want to hop the services but layer security instead. Maybe that's too old-fashioned. Hope there will be Signal Protocol-based addons/libraries like this one day.
I live happily without Facebook, but I'm not a teen anymore.

I was thinking: is HN a social media?
Even then it's just an arms race. Someone will reverse engineer the app and figure out how to encrypt that block of data.
> It is however interesting that we used relational databases for virtually everything for decades even though SQL is suboptimal at most things

You have no clue what SQL or ACIDity is. For 99% of the cases SQL/RDBMS is the right choice. You probably think you belong in that 1%, but from your comment, I suspect you do not.

> I choose to use NoSQL for virtually all my projects

That's because you have no important data to store.

When you get to store data that are important to your customers you're gonna have a big revelation.
I find this very confusing, but I guess I'm not the intended target audience. Not that I say it's wrong, but I don't really see the point.

Do people really expect R^2 to measure the fit of the model to the true model? R^2 measures the fit of the model to the data: i.e. how well does the model perform in predicting the outcomes. In his first example is clear that all the models are equally useless: the noise dominates and the predictive power of the models is close to zero. In the second example the predictive power of all the models has improved, because there is a clear trend. The true model predicts much better than the others now, but each model predicts better than in the previous example.

In the first example, he concludes: "Even though R^2 suggests our model is not very good, E^2 tells us that our model is close to perfect over the range of x."

Actually our model is "better than perfect". The R^2 for the linear model (0.0073) and for the quadratic model (0.0084) is slightly better than for the true model (0.0064). Of course this is not a problem specific to the R^2 measure (the MSE for the linear and quadratic fits is lower than for the true generating function) and can be explained because the linear and quadratic models overfit. E^2 is essentially the ratio the 1-R^2 values (minus one). We get -0.00083 and -0.00193 for the linear and quadratic models respectively (the ratios before substracting one are 0.9992 and 0.9981).

In the second example,"visual inspection makes it clear that the linear model and quadratic models are both systematically inaccurate, but their values of R^2 have gone up substantially: R^2=0.760 for the linear model and R^2=0.997 for the true model. In contrast, E^2=85.582 for the linear model, indicating that this data set provides substantial evidence that the linear model is worse than the true model."

The R^2 already indicates that the linear model (R^2=0.760) and the quadratic model (R^2=0.898) are worse than the true model (R^2=0.997). The fractions of unexplained variance are 0.240, 0.102 and 0.003 respecively and it's clear that the last one performs much better than the others before we take the ratios and substract one to calculate the E^2 values 85.6 and 35.7 for the linear and quadratic models respectively.

(By the way: "we’ll work with an alternative R^2 calculation that ignores corrections for the number of regressors in a model." That's not an alternative R^2, that's the standard R^2. The adjusted R^2 that takes into account the number of regressors is the alternative one.)
Travis-Ci Local Docker Images
My main issue with R^2 is that it is an artificial indicator. It has no business meaning. The best alternative is to build a metric that is related to your business issue. How many $ do you loss if you're wrong by 1%? Or by 10 units? Do your business loss is linear in your error? Is it symmetrical?

These are the kind of questions you have to ask yourself. Defining a metric is hard, and there is no good shortcut.
The MapCell response actually returns "NearbyPokemon" and "WildPokemon" separately - where that node implementation is using the "WildPokemon" list. I mostly just think just using the "NearbyPokemon" which returns distance in meters only, not latlong, and displaying only the steps would be more in the spirit of the game is all.

The Fort/Gym data from the map request returns only the top cp pokemon in the gym. But the actual gym data returns the actual list of member pokemon, which might also include the nickname (POGOProtos definitions include it, but would need to make an actual request to confirm).

So maybe!
BSI – Industrie Und Kritische Infrastrukturen  Meldepflicht
I just made a little iOS game too! Actually this time I managed to stay on track and get everything done in about 1 month. I honestly don't know how I did it, but it's really fun to finish a project. http://boopsboopsswoops.com/
Command Line Interface for GitHub Additional Features
Navier Stokes simulation in OpenCL
I didn't mean to imply grotty at all, and apologise if I'd left that impression. I was just noting that this had the hallmarks of LaTeX to HTML translation.

Also the index.
And React news? Python news? Maybe we can just have HN tags :)

I could be reading too much in your comment but I get the feeling you're upset at the overwhelming amount of pokemon-related threads. Don't be.

Reverse engineering is a very interesting field of programming. Video game reverse engineering is one of the most amazing things, actually. It's where many teenagers discover how the games they play work. Where they first start understanding technology, write their first scripts, start being in control of their computer.

I haven't touched Pokemon Go yet, and am not overly interested (mostly because it's not available where I live). But the reverse engineering effort behind it is spectacular and passionating. Why dismiss it like this?
Yes, and I've purchased and read books, and I've read online resources. Granted, none of this has been within the last several years, so if there have been major changes in that span I've missed them.

But thanks for the charitable question!
In much of Asia generally, income disparity is such that middle class-ish people can and do hire staff for a lot of things that they couldn't do to the same degree in the US or Western Europe. Thus things like taxis are cheaper.

And Hong Kong is relatively expensive. My experience in southeast Asia is that, in some places, you can hire a car and driver for the day for maybe $50 or so. So--combined with how horrible driving in the big cities there are--many people just have drivers.
New npm package – flying drones with REST
It is an absolutely straightforward example of a racist comment. Trump knows full well that his complaints about Curiel are baseless, so he's lashing out by trying to make something out of the fact that the judge has Mexican parents. In other words, he's using racial insults because he's lost the argument. In legal and logical terms, he might just as well have said that Curiel is a poopy pants.

Quite often there are efforts to weave conspiracy theories around Trump's incoherent ranting. So in this instance, there is some kind of story about how the California La Raza Lawyers Association is a shady "ethnoactivist" group, whatever exactly that is supposed to mean. All of this is just a way of providing cover for another one of Trump's uncontrolled bigoted outbursts. The judge is a member of a completely unremarkable kind of law association. It is no different from a gay judge being a member of an LGBT bar association, or a black judge being a member of a black bar association. Everyone knows that these sorts of organizations have the purpose of tackling specific problems faced by (respectively) LGBT and black people in the legal profession. To say that membership of such an organization can disqualify a judge from a particular case is effectively to say that gay judges can't make judgments on gay issues and black judges can't make judgments on black issues. That would be racism/homophobia par exellence.

By the way, you may recall that Trump has expanded his comments on judges since. He also believes that he could not be judged fairly by a Muslim judge. I wonder what complicated theories people have come up with to try to justify that comment.

It says something quite extraordinary about the present state of the country that there are people willing to defend a Presidential candidate who attacks a federal judge using ugly racist language. Even mainstream Republicans like Paul Ryan don't want to touch this.
Ask HN: What is the price of privacy?
After the recent attacks in Europe, which seem to happen with an unfortunate regularity, I left wondering about the role and price of privacy. I've seen that many (most?) of them have, in hindsight, provided more than trivial clues about what was happening. In some cases the communications have been unencrypted! So the question arises: Why was not these people under surveillance? Wouldn't that have been a win for the society?

I've been an advocate of Bitcoin (and made some toy projects), as well as being interested in privacy-friendly technologies (PGP, Tor, etc.). But thinking about the subject, these technologies (as well as privacy-friendly, 'censor-resistant' hosting like pastebin) are a nest of problematic people (and not "problematic" as in the V-for-Vendetta romantic sense, but really messed up, dangerous people who make things we refuse to believe are true).

I won't hide the fact that I am writing this post behind TOR. You could say that that's exactly the reason why we need it. But I don't think so. I just want don't want to make it trivially linkable to my accounts, but I would be perfectly fine with proxying that through a state-controlled proxy. For this post, I want privacy from random eyes, but not from the government.

I'm starting to see the government as the Operating System of the society. If you don't trust it, you already lost. On the other hand, if you trust it, it protects you against external "malware". We -as a collective- used to frown against closed technologies like TPM, walled gardens, etc., but they are a major wall against casual malware distributors. Of course, not without drawbacks, and that's the question, if the whole pack is worth it.

We… [limit of 2000 chars, continue at http://pastebin.com/NPWLJLTe ]
'Permutation City' by Greg Egan, a Science Fiction novel, has as one of its main subject a variation of this idea.

Recommended, it's a very interesting reading.
Racket v6.6
I rtied ti nad my lysdexia gto crued. Sussecc!
One of the issues with hydrogen power that you can't use regenerative braking as easily. Of course you could run a hydrogen/battery hybrid system, but its not obvious that hydrogen would really be that beneficial in that configuration.
Well, in that case, they probably really are working for Killary.

Damn, I thought google was one of the good guys.
Seriously, you are critizising a technology that is not there yet in comparison to a technology that is also not there yet? Recharging electrical cars in a smart grid, long range capability (we all know what 200 miles in a Tesla means and how much of it is left after a few years), loading time etc. Are they solved right now?
And these are real problems, not "having to go to the gas station as all the other people" problems. In comparison to gasoline powered cars are hydrogen based cars mostly missing infrastructure and mass production. Electrical cars do too, but you cannot say one is better than the other as they are pretty much complement to each other.
Assume it's a team of 3-8 eng/ops/Devops managing each complex tech on the list, then 5-10 more teams of 5-20 engineers managing areas of the product, and interfacing with the infra teams to get things done.
Upvoted for super-controversial and entertaining idea.

This would be the ultimate case of "When your model don't fit your population make your population fit your model".

Genetically engineered traits that discourage rebellion and foster cooperation sounds scary to me. Outliers could get even more power in that kind of society. I can see the headline "Scientists combine Swedish consensus seeking genes, Japanese Wa-genes and Labrador retriever genes to create new super cooperative Goofy-Lagom-Harmony Sapient race in Mars."

https://en.wikipedia.org/wiki/Wa_(Japanese_culture)
> you should NEVER TRUST USER INPUT

I think a lot of people are reading this and think that this advice is too wide, and include so much in so few words.

It's the same caliber as "don't trust strangers" and "be responsible of your actions". There is undeniable truth in it, you cannot go wrong following it and it's difficult to argument against. But I think that's what makes it counter productive and basically devoid of meaning.

Effectively if the rule is as old as computing, but it still has to be voiced, it means it's not a simple rule to begin with.

As you put it "you must understand how it's used and filter, strip everything that is not needed away". This basically means every time you have user input, ideally you'd have to audit all the libraries and frameworks accessing the data to check how they use it, and filter accordingly. Pass a construct to a json library ? first check what the json library does with it, sanitize your input for everything that could be harmful to the library. This solution is voiced in one sentence, but would mean hours/days of library auditing in a real world scenario.

TL;DR: you can't cover for everything, you have to choose your battles. Knowing which libraries have known vulnerabilities is valuable info.
> And that is to some degree the beauty of hydrogen: you can store it relatively well for some time

I thought storing hydrogen was challenging, since hydrogen atoms are so small?
That's what makes this something of a paradox. It's far more likely that we are experiencing an illusion in the heat death of the universe than we are experiencing the "original" universe.
I ran numbers on energy equivalence of fossil fuels to wood in a recent comment. Very roughly, a ton of coal or barrel of oil equals a cord of wood (actually somewhat more, but close enough).

Given US timber production and multiplying by five to get an estimate of global timber ... estimated fossil fuel resources are equal to about 200,000 years of global timber production.

That's a lot of wood.

Even if you're only looking at a small recoverable portion, say 10%, it's 20,000 years.

Energy translates reasonably well to carbon. There's some allowance for natural carbon sinks. But truth is we're messing with the carbon cycle in a huge way.
Mr. Manning now has enough fueling options in Southern California to cover his 45-mile commute to Playa Vista with little anxiety.

Wow!
Ruby on Rails and the importance of being stupid (2009)
After reading this, The Last Question by Azamov takes on new meaning.

http://multivax.com/last_question.html
Private Cloud Computing: The Enterprise Preference
You don't know who the user is until you verified the integrity of the JWT. Verifiying the integrity requires the secret. Your solution adds the dependency: the secret requires the user. It is cyclic, unsolvable without breaking a constraint.

You could assume the username is correct, then get the secret, validate. But that sounds like something breakable.
If you just want a quick proxy to inspect traffic, apache with mod_dumpio¹ always seemed the quickest and easiest way to do it: just proxypass your traffic and

LogLevel dumpio:trace7
DumpIOInput On
DumpIOOutput On

and all your traffic is in the log files.

¹http://httpd.apache.org/docs/current/mod/mod_dumpio.html
The thing is that having a tool like this does not make the requirement gathering go away.

Dude I can build a house. I've only used a drill once in my life, I've never mixed concrete, or bricks, but I can. I will lay stuff on top of each other and use superglue and whatever it takes.

But nobody will ever want to live in and it would not be safe. That's why you get a professional to do it.

A user might somehow be able to magically skip gathering requirements and just dive gung ho into creating a structure for what they want. Great. And you end up with the legacy Excel monstrosity turds that we've been trying to eradicate from the Enterprise for the past two dozen years.
The Transit Ridership Recipe (2015)
Security and convenience exist on a spectrum.

For my ordinary checking account, I opt for convenience. I don't want transactions randomly declined and I don't want to have to wait for banking hours to authorise activity. To compensate, I limit the amount I keep in the account.

For certain other accounts, I opt for more security. Cheques are blocked; foreign transactions are, by default, blocked; online banking must be two-factor authenticated every time; transfers must be authorised with a phone call below certain amounts and in person at a branch, with ID and a passphrase verified, above certain amounts; et cetera. These are flags one can have enabled on most bank accounts. They're just debilitatingly irritating for ordinary use.

If you make banks responsible for user-authorised fraud, e.g. a customer wiring money to a scammer, you're also asking them to nanny you. Freedom and protection from your own stupidity exist on a spectrum.