skysight-inc/HackerNews huggingface dataset:
I used to work on textual Information Retrieval (which is quite related to search). One of the key contributions of my PhD thesis was a statistical method to extract concepts from texts [0].

Surprisingly enough, with those concepts, Tf-Idf [1] was quite good to extract keywords from documents which allowed us to build document descriptor tables which can eventually be used for document search [2]. We also built a small prototype that allowed us to retrieve results of specific areas of documents where the searched concepts were more "valuable" than on other parts of the document [3].

It is a very interesting subarea of AI/NLP which unfortunately doesn't seem to attract much interest.

Since the article also talks a bit about Wikipedia dumps datasets, here's a tool that I've created to build textual corpora: https://github.com/joaoventura/WikiCorpusExtractor

[0] - http://www.sciencedirect.com/science/article/pii/S1877050912...

[1] - https://en.wikipedia.org/wiki/Tf%E2%80%93idf

[2] - https://link.springer.com/chapter/10.1007/978-3-642-40669-0_...

[3] - https://link.springer.com/chapter/10.1007/978-3-642-40669-0_...
Sure, it could be used elsewhere. OTOH 7763 calls out email as a use case three times and in section 5 Examples "email attachment" is the only example.
It was stated in the keynote that the numbers are much better (an order of magnitude) for face versus fingerprint.
I don't believe resolv.conf is in any DNS RFC individually.

Here's a list of all RFCs related to DNS: https://www.isc.org/community/rfcs/dns/

In https://tools.ietf.org/html/rfc1034 this is discussed:

The resolver always starts with a list of server names to query (SLIST).
This list will be all NS RRs which correspond to the nearest ancestor
zone that the resolver knows about. To avoid startup problems, the
resolver should have a set of default servers which it will ask should
it have no current NS RRs which are appropriate. The resolver then adds
to SLIST all of the known addresses for the name servers, and may start
parallel requests to acquire the addresses of the servers when the
resolver has the name, but no addresses, for the name servers.

I believe it would be an accurate statement that a DNS resolver which is incapable of servicing all DNS queries by the client is something which many clients are not designed to expect. But relying on the behavior of glibc in the way it attempts lookups is in essence relying on the internal workings of that software rather than anything written in the spec.

BIND9 supports 'views' (iirc) which allow for splitting queries between internal vs. external DNS. But that would be the resolver as opposed the users desktop implementing that.
> Better tools to filter out trolls leaves more space for healthy debate.

The very nature of US political discourse itself leaves no space for healthy debate. I'm sceptical that removing the trolls is going to be sufficient to fix that.
You can mean what you want, but in this case you're using an important idiom from philosophy to mean something else. People will pick you up on it. Best to say "raises the question" if that's what you mean.
Equifax compromise and their insecure response
Coupling of pollination services and coffee suitability under climate change
There's kind of a lot of issues. Especially with psychology and sociology and the areas they intersect.

1. Replicability of social psychology results are estimated at 25%. (Cited in article, from http://www.nature.com/news/over-half-of-psychology-studies-f...)

2. People are not reporting non-significant results. Which means we get an incomplete picture of the validity of certain hypotheses. (from article)

3. The numbers don't line up when comparing observed power and significance of results. (from article)

4. Some results data is erroneous or falsified. The GRIM test that surfaced a while back shows numeric "results" that cannot possibly be derived from the sample data. (https://medium.com/@jamesheathers/the-grim-test-a-method-for...) (https://www.vox.com/science-and-health/2016/9/30/13077658/st...)

5. Many authors on the papers don't want to share the data from their papers - even if it's a condition of publication. (from a previous article about the GRIM test that I can't find) This doesn't directly equate to maliciousness or deceit, but it illustrates incorrect attitudes in the field. Some authors consider it undignified to have someone "check their work". Others are concerned of fallout if their data contains errors. (Generic mainstream blog/press article on reproducibility https://www.vox.com/2016/3/14/11219446/psychology-replicatio...)

6. Ridiculously small sample sizes are often used in experiments. Often this doesn't seem to affect how the results are received, though it should. (briefly discussed in article and comments)

7. Papers and studies are cited in articles and future papers before they've been peer reviewed, replicated, or proven sufficiently to justify their inclusion into the field. Thus the results are "baked in" to social psychology as fact at an uncomfortably early stage. (http://blogs.plos.org/mindthebrain/2015/12/30/should-have-se...) This kind of feeds into the hype machine where people focus on significant results - even if the data needs massaging to get there. Even after being refuted (if refuted) the field and its fans never seem to fully recover and back away from bad science. By then it's already made its rounds on Facebook and it's even referenced in the worst of the college intro to psych/sociology courses.

8. Journals in general need to be more open with their data. Especially with publicly-funded research. Paywalls don't help progress.

9. Corporate/politically funded white papers publish studies without making the connections clear.

The field needs a massive overhaul in the sense that its participants need to stop chasing controversial results in the press and instead focus on reproducible results and methods. All fields need some of this, but psychology needs it more than others.
I mean, it's less plainly visible then your face, or even pictures of your face.

I bet you there's an algorithm somewhere that can take a picture of your face and turn it into a 3d model. Then you can take that model, 3d print it, then use it to unlock your phone.
Made me giggle that the ad for this is some surfer being totally in the middle of an amazing moment and then getting distracted by a phone call.

I really like my apple watch, but this really isn't selling it...
I wonder if there is a technical reason for that or if they aren't ready for possible cannibalisation yet.

I was really hoping it wouldn't require a phone at all.
> "That is a certainty" doesn't cut it for me. That's merely a claim or thesis, not a statement of fact, not necessarily wrong but not necessarily true or the whole picture either.

We don't need to provide evidence or the complete picture for everything. If I say 'That is a certainty' that London has a larger population than Dover, it's not necessary to provide a citation or a detailed picture. The existence of discrimination in the areas I mentioned easily falls into that category. I didn't say it was the whole picture; that would require writing books, not HN comments.

> Same for your other claims.

Almost all my other statements were reasoning about data.

> What I find interesting is you instantly leaping to "discrimination/oppressions explains it all".

I didn't say that and I didn't leap to my conclusions; I'm not sure how you imagine to know what my thinking process was or how long it took. I saw data, shared it, and reasoned a bit about the evidence.
The really hard problems in this area aren't technical. They're social. Any attempt to make comments sections useful again is a welcome step.
So, basically, you want to outlaw most current implementations of Level 2 autonomy, which are being shipped by a large number of carmakers.

I was unaware that Waymo was planning on shipping a Level 2 system.
Anyone else experience major issues with the website? It's inscrutable to me with the scroll based parallax nonsense breaking horribly.
> the more questions we ask, the more questions we have to answer, but also that many of the 'easy' hypotheses have been mined out.

Like a tree? The main branches are the most visible, and so the most tackled. The smaller branches are only realised when you're deep enough in it. That's where our age's advantage comes in, as we have many more qualified 'explorers' than before, but it also makes it all the more important that the foundational branches are solid.

But then again it may be closer to readjusting our lens. The big questions (of the natural kind, not innovation like AI) today still pertain what we take for granted, like quantum mechanics is making us question the nature of reality again. But much like the discovery of waves once threatened the credibility of light as particles, I'm optimistic that reconciliation can happen, and to ascend to another level ... but where can we stop? When can we say "This is it"?

Just musing..
Not always possible if law enforcement is breathing down your neck.
> These keynotes were magical before leakers leaked everything

Even without leaks, there's usually nothing really surprising in the keynotes. You get the usual hardware and software upgrade you would expect from a year to the next. Far from magical, I find these keynotes stereotypical. The fake enthusiasm, the marketing buzzwords and the endless flow of superlatives are sometimes painful to watch. I almost feel bad buying their products after seeing this (unfortunately they still make the best laptops!).
So Atom is yet another editor supporting the language server protocol. Whether you like Atom or not, that's good news. The more that editors support it, the easier it is for new languages to get a good IDE experience with many editors.
If I was given an infinite supply of free heroin and all the time in the world to sell it to my neighbours, friends, family, workmates and anyone else in the locality then I would struggle to make a sale. My client base would be zero and I would have to go back to the day job.

If you rewind America to before the opioid epidemic then that would be the situation for anyone aspiring to sell heroin in normal America. The demand would not have been there, you would need to get a sizeable community onto crack cocaine before they need heroin too.

The prescription drugs and how that worked out has changed the demand side of the situation. So it is not a matter of some external evil drug kingpins dumping vast quantities of cheap heroin onto the innocent U.S. market, it is more the vast quantities of Americans doing whatever they can to get their fix.
Well, I hear a lot of people complaining that the results on DuckDuckGo are still worse than on Google, even though both search-engines produce results within a second. And these are people that really want to quit using Google for privacy reasons. I never hear people complaining that a search is slow. So I do think that search-quality is where the competition is happening.

Edit: But, I agree, we don't often see any good HN posts or papers about search-quality. Perhaps a case of trade secrets?
And now github is using the same license too :)
Tupress, the new JavaScript Framework
Super 8 Languages for Making Movies
Email and JS = heart
Wyatt from the Coral Project here. It's all about the news rooms owning the data of the comments, not social networks like Facebook. We're using tech like GraphQL to make the experience possible on multiple platforms through our extensible and pluggable API.
The car makes it clear to drivers what "autopilot" means. The only people who might be confused by the term are non-Tesla drivers.
Damn. Guess I'll have to stick to flagging them. Thanks for the info though.
It's easy to assume the worst of researchers, but I think Gelman's 2013 "Garden of Forking Paths"[1] does a good job of showing how it doesn't have to be intentional misconduct. It more likely or more often comes down to implicitly selecting your analysis plan given the data, and not correcting for the (hidden) multiple comparison problem that arises when doing frequentist statistical tests in that reality: the theory behind null-hypothesis testing and p-values dictates you account for all the other analytical choices you could have made for that data, or would have made in the face of other datasets.

Gelman's other point is this is all made much worse in inherently low-power fields like psychology and the social sciences. A field like physics can "get away with" it more because making the appropriate corrections is less likely to wildly change the conclusions.

[1]: http://www.stat.columbia.edu/~gelman/research/unpublished/p_...
Agreed, I like the notch. Mind you, I don't even own an iPhone, but I also like the Essential (Android Phone) notch. The Android phone's notch was less wide, sure, but I'm not sure it matters.

By all means, if you're displaying UI stuff there then great, don't obscure your buttons with the notch. However I love seeing images/etc with all possible screen visible. I've been wanting a head/chin-less phone for years now.

From the looks of it, I love both of them. The Android and iPhone version.
There can be a commission at the end or there can be an extra % to be made. When I was buying and selling in BTC if I needed to buy or sell BTC I would trade on localbitcoins, buying at 5-10% under the market rate and selling to the market 5-10% above market rate.

Further, while the transfer via the bank is usually a flat fee the bank also make a cut on the exchange rate that they charge the user, so there is a % element to consider on larger transfers.
Not sure how you could ever hope to have productive debate on say an article about global warming. It's not a local topic, it's a difficult topic, there's millions of regular people and trolls around the world waiting to blab about something they don't have any clue about.

It's hard for me to imagine what a comment on a Washington Post article could add to the article or to anyone's knowledge. It's user involvement for the sake of user involvement. "Healthy debate" is maybe unobtainable at that scale.

HN can get pretty ignorant and off in the weeds (hey we're all guilty) fairly often, and that is despite being topical, relatively small, having relatively informed users and active moderators.
This is a language for video editing. The language can be found at: http://lang.video
There are, roughly, 27 Portlands, in the U.S alone.

https://en.m.wikipedia.org/wiki/Portland
> The most powerful and smartest chip ever in a smartphone, with a neural engine that’s capable of up to 600 billion operations per second.

sorry but "neural engine" is just too funny. i mean there's normal levels of bullshit bingo and then there's branding your chip "neural engine".
If the Equifax leak and the concerns around not being able to change ssns teach us nothing, it's this.
Oh I agree with you fully, when I posted this comment there was nothing here. Just wanted to clarify a bit for people.
I flagged your comment to help get the mods' attention.
That's quite different if I'm reading your meaning correctly. That type of system tends to result in the biggest threads staying at the top and getting bigger and bigger while new ones never appear.
Ask HN: What's your favorite history of science book?
I love to follow HN recommendations about CS books and I wonder what you can suggest to read in the genre of history of science.

I only read "Taming the infinite: The story of Mathematics from the first numbers to chaos theory" by Ian Stewart and it was really fascinating journey through mathematics. I only wish this book to be longer. Please post your recommendations for any kind of science.
>And if you want really go all out, most phones — including iPhone — support 5 or even 6 digit passcodes.

iphone supports arbitrary-length alphanumeric passwords.
Linux distro maintainers are making the decision to use the systemd replacements. If you disagree, use a Linux distro that doesn't, or make your own.
Self-signing is a great way to invite phishing attacks against your end users: if they're used to clicking through cert warnings all the time, they won't notice when something has actually gone wrong.

Running your own CA works but has two caveats: first, I don't really trust myself to run a CA competently given that almost nobody who runs a CA as their only business function is doing so competently, and second, you have to get your config onto every device on the network. If you let personal devices on the network or even just hard-to-configure things like Chromecasts, you have to get your config onto all of them somehow.

It's quite a bit simpler and more secure to just use .corp.example.com or whatever.
>using registered trademarks //

How is a common description of a standard breakfast cereal still a trademark? If granola is an RTM then I'm afraid it's become generic (in UK at least, it's just "what US Americans call muesli").

Wikipedia have it as an RTM only in Australia and NZ.
Whether or not this specific example is, I know people who legitimately believe this.
I have bought way more drugs with fiat than I have ever done with bitcoin.
I think it depends on the amount of beard you shave off.

Worst case is that you have to use the passcode and it'll relearn your beardless face.
Yeah? Did people need to prepare for the y2k bug in 1979?
I disagree.

The attackers can't get the data if you don't store that data to begin with. Voila!

I think there should be legislation that simply restricts who can store which data, as well as legislation that forces open-sourcing of critical infrastructure software.
I have a question. If one were to believe batteries or EV's are our future, what kind of business or startup opportunities does it present?
Except that we already have Jetbrains for this.
Obviously the choice is going to be between complying or a holding cell. This allows you to make that choice instead of having it made for you. That's not irrelevant at all.
None. I just imagine it's useful that other people understand you.

But hey, if you want to use the wrong words, phrases, etc, then it's your right to do so. ...
You don't get to decide the future! Apple gets to decide the future. Notches and dongles for you.
Sextants.
The difference, of course, is that while a SCSI drive is out-of-date, a headphone jack is not; it is, in fact, superior in every way to a Bluetooth audio connexion.
Well I guess that's a matter of theology.
iPhone 10 processor is about as fast as a quad-core Intel Kaby Lake Core i5
and the $60 xiaomi 4K TV box has been around for a year...
> I imagine at a place like Google, your "corpus" is just about everything under the sun, as are your queries.
> What happens then?

You accept the fact that a good search engine is inevitably going to be a multi-layered beast that requires constant effort to improve, with heuristics, machine learning, magic constants, special lists of words, and regular expressions.

After that, it's just a matter of picking a commercially useful objective function ("instant search", "query autocomplete", "maximize user satisfaction") and pointing large teams of well-resourced PhDs at it. Probably a whole weekend at least.
I don't have any patents why should I give a fuck about people who want to steal money from facebook?
> I feel a bit like I'm being trolled

Sorry, not my intent at all.
I wonder if doing it app by app makes more sense than splitting the OS. So an app can be phone aware and desktop aware and so on.

This already sort of happens with some web apps.
Not entirely true. Your email is your password. If you own email and access to it, you can reset and request new password.
Curious why you come to that conclusion? That study itself says the watches were within acceptable error for their purposes. Why would you need more accuracy than the roughly 2% error on the Apple Watch?
If those two are using SHA-1, like git, you are still using a standard

Lovely, so I can officially claim I use a standard that is required by many commenters here if I just hash random bytes! Should I use a NIST-approved standard or maybe GOST hash function will work? Can I use the output of ChaCha20? If so, I'll be happy to read 16 bytes from /dev/urandom.

The third one however sounds like a potential mess.

Thanks for your analysis!
Makes sense. What operating systems do you need this for?
This latest keynote was pretty much completely spoiled due to a leak which came internally from Apple (URLs to golden master iphone firmware). I was really surprised that happened. I follow Macrumors pretty closely so I'm used to seeing a lot of hardware leaks over the years from manufacturing partners, but this fall's leaks were kind of insane.
I don't know what you mean by "marketing". I am mentioning technical facts; you are saying things like "systemd tries hard to do stuff it doesn't really understand" and "reality distortions." Can we have a technical conversation?
I don't, given that core claims in the letter, which is clearly drafted in anticipation of litigation by legal, are contradicted by the reporting.

Maybe you just get a sense of when a lawyer making shit up and slinging shit after you've put together a few dozen similar letters.
Apple's DACs are excellent. Here's an analysis of the Lightning adapter, for instance:

http://www.kenrockwell.com/apple/lightning-adapter-audio-qua...
The problem is so much bigger than that. Here in Edmonton, we have regular folks who campaign and crowd-source funding to install lights on bridges, buildings, just about anything for the sake of making things look 'nicer'. Which is unfortunate for those of us who want to see the sky.

Anyway, how's the light pollution been on your Africa trip? You must have been through some seriously dark remote areas by now.
The entire game is role play. You're not actually piloting a ship. You're not actually part of an alliance. In a sense, there is nothing to betray outside of the role play.

Of course, I understand what you're really describing. It's just amusing that you and I don't imagine normal behavior to be role play, and can conceive of a meeting wherein people bring their Eve personas together in a simulated star system and agree that these characters will then engage in a recursive level of role play.
LTE is a great addition but disappointed Apple Watch 3 has no new sensors. See you next year Apple Watch!
Have I implied otherwise ?
I've never encountered an "Emoji Legend" before. I instinctively skip over the word "emoji" so I missed that section.
Social Facilitation – Why we perform differently with or witho an audience (1965)
They have a Bosch supplied radar but insisted on doing their own signal/object processing.
I would love to see a big site try to use Slashdot's old commenting system. Commenting felt like a privilege and so did moderation.
Can we call that notch what it is please, a monobrow.
> anyone can do anything to anybody else at any time

That isn't libertarianism at all. Libertarianism is you can do whatever you want, provided you do not employ force or fraud against others. There's more to it than that, but that's the core idea. Another way to say the core idea is people have inalienable rights - life, liberty and pursuit of happiness.
True. If you are running local, not via a terminal, it should be much faster. I've never seen either the 6800 or the 2901 models in person.
Oh, ha! In the winter when wearing gloves or if my hands are wet, I often swipe my phone open or click to answer a call using the tip of my nose. Guess that's not what this advice is about, though I was briefly happy to think sufficiently many other people had this habit to warrant an cautionary article.
> But when they cause more problems than they solve

I'm willing to bet traditional payment card anti-fraud and consumer protection systems are solving more problems than they cause.

So you have been inconvenienced by them on occasion. That says nothing for the frequency these systems work as intended.

> The backup is simply a list of 12 words.

I wasn't aware of that, thanks for clarifying.
I was pissed off at my projector doing that (same problem with a different native resolution), but then it was a $90 projector. Sorry for your TV.