skysight-inc/HackerNews huggingface dataset:
Extract:

> In the weeks after the Pausebuster paper was published, Dr Bates conducted a one-man investigation into this. His findings were extraordinary. Not only had Mr Karl and his colleagues failed to follow any of the formal procedures required to approve and archive their data, they had used a ‘highly experimental early run’ of a programme that tried to combine two previously separate sets of records.

This had undergone the critical process known as ‘pairwise homogeneity adjustment’, a method of spotting ‘rogue’ readings from individual weather stations by comparing them with others nearby.

However, this process requires extensive, careful checking which was only just beginning, so that the data was not ready for operational use. Now, more than two years after the Pausebuster paper was submitted to Science, the new version of GHCN is still undergoing testing.

Moreover, the GHCN software was afflicted by serious bugs. They caused it to become so ‘unstable’ that every time the raw temperature readings were run through the computer, it gave different results. The new, bug-free version of GHCN has still not been approved and issued. It is, Dr Bates said, ‘significantly different’ from that used by Mr Karl and his co-authors.

Dr Bates revealed that the failure to archive and make available fully documented data not only violated NOAA rules, but also those set down by Science. Before he retired last year, he continued to raise the issue internally. Then came the final bombshell. Dr Bates said: ‘I learned that the computer used to process the software had suffered a complete failure.’

The reason for the failure is unknown, but it means the Pausebuster paper can never be replicated or verified by other scientists.

...

‘I want to address the systemic problems. I don’t care whether modifications to the datasets make temperatures go up or down. But I want the observations to speak for themselves, and for that, there needs to be a new emphasis that ethical standards must be maintained.’

...

Last night Mr Karl admitted the data had not been archived when the paper was published. Asked why he had not waited, he said: ‘John Bates is talking about a formal process that takes a long time.’ He denied he was rushing to get the paper out in time for Paris, saying: ‘There was no discussion about Paris.’

He also admitted that the final, approved and ‘operational’ edition of the GHCN land data would be ‘different’ from that used in the paper’.

As for the ERSSTv4 sea dataset, he claimed it was other records – such as the UK Met Office’s – which were wrong, because they understated global warming and were ‘biased too low’. Jeremy Berg, Science’s editor-in-chief, said: ‘Dr Bates raises some serious concerns. After the results of any appropriate investigations... we will consider our options.’ He said that ‘could include retracting that paper’. NOAA declined to comment.
Respectfully, the APL of today is not the APL of the 80's. APL is tremendously well suited to many of the problems that face other programming languages, such as the parallel programming usability problem. Modern dfns based APL has incorporated lessons learned from languages liked Scheme (Lisp) to make function syntax and control flow more functional.

And APL has been and continues to be used to solve problems from web programming to graph processing and everything in between. While it finds immediately obvious advantage in numerically dense calculation heavy code, my research is specifically interested in demonstrating and scaling APL beyond the "status quo" of what people think APL is good for. The Co-dfns compiler mentioned in these threads is precisely such a counterpoint, demonstrating how to leverage APL in a domain space traditionally dominated by OOP and Lisp style languages, and more recently by certainly highly typed languages such as Agda or maybe Haskell.

The difference is that good APL is often more or less "parallel ready" out of the box. Good code in other languages is often anything but. I am working through making the case that "parallel first" languages such as APL are actually pedagogically and programmatically better than "serial first" languages. To this end I have conducted at least one exploratory pilot study on the matter:

http://video.dyalog.com/Dyalog13/?v=kIItfQJEVdM
This one is a bit of a classic, for the last 8 years (apart from in 2015) it has been on HN at least once a year:

https://hn.algolia.com/?query=TTY%20demystified&sort=byDate&...
Iirc the whole chain needs to be licensed and copyright transfered, that's to avoid issues with people laundering pirated items with a legit contract knowingly.

So the issue remains if they knew the wendir hadn't a license - that'd be necessary to know where to assign damages, but still they would not be able to use the output, as created without a license to begin with.

If the vendor had not a license to work with the software the output has been illegaly obtained, while claiming copyright on the output may not stick claiming a royalty along punitive damages is not that far fetched.
While I agree that interviews have high variance and a lot of times the interviews don't test what's required for the job, mini projects are a huge time sink for candidates too. Most candidates aren't going to spend a couple of days working on a project just to apply to a company. Unless all companies decide to just switch to using project-based interviewing, interviewing is here to stay (despite its quirks).
The extra context I'd add to your statements is that poor choices of abstraction are overwhelmingly more common than good ones, and that insiders learn to live with them.

There's definitely a tendency for that to happen, I agree. If nothing else, we often don't know what good abstractions will look like yet in the early days of a project.

Even if we do, the most useful abstractions might change as the project evolves. Fortunately, the kinds of abstractions that tend to be most useful for hiding a lot of complexity also tend to be quite stable, in the sense that while they might be extended or generalised as the project develops, it is rarely necessary to severely break backward compatibility.

In an ideal world, we could definitely have more natural tools for things like refactoring, source control and diffs/merges, though, to lower the kinds of barriers you mentioned and promote willingness to refine the design and keep it clean as a project develops.
> you can't represent all valid HTML documents as S-expressions, at least not in the convenient way people assume

Of course you can. Here is how to express your example as an s-expression:

((:!doctype html)
(:title "This is the title")
(:p "..."))

Here it is being rendered by CL-WHO:

? (princ (html ((:!doctype "html")) (:title "This is the title") (:p "...")))

<!doctype "html">
<title>This is the title
</title>
<p>...
</p>
كيف اعرف برجي الحقيقي – برجك اليوم ماغي فرح
"Take a break!"
I think the criticism lobbied against AirBnB and Uber is for a reason, and the reason isn't the same "up-yours" attitude of past maverick tech companies. It's a little easier to buy the maverick attitude of something like gnutella when the author isn't getting close to billion dollar revenue yearly. It may seem a little strange, but the ethos is a bit easier to believe when you stay outside the system and benefit everyone instead of just looking out for yourself alone.

The reason Uber and AirBnB are the subject of scrutiny is simply because the difference between them and any other business is non-discernable except that they have the money to stall out legal issues. Just from a "feel" perspective, they don't really feel like they're fighting the fight for everyone, they're just trying to line their pockets. There's a world of difference between what they do (offer a service at cost) to something like bit torrent or gnutella.

I can't really say where or what the limit is on something like this, when a software maverick stops being a maverick and starts playing the game, but admittedly I haven't really carefully analyzed it beyond just "here's why I see a difference". Uber and AirBnB (and quite frankly, many other SV start ups) don't really have that hacker feel to them like we'd see with older projects - instead it feels much more like they're rooted in business and business alone.
I wouldn't say there are any new insights per se; I'm maybe just more intimately acquainted with them. Ie when I first read it, most of the descriptions about things like how a paycheck influences your thinking/behavior were just intellectual ideas for me, but having had a mortgage + paycheck for 4 years I've come to personally witness how it's 'domesticated' me in a way, made me more fragile, however you want to describe it
While thinking about this for a bit after reading https://news.ycombinator.com/item?id=9419930, I thought of something:

Why not just get the home package and run it through your own overlay system? The fatal flaw here with this design is that there's nothing in place of the glass. With enough tinkering (a pub subscription and a home subscription to play with, and maybe a fortnight to a month, if that) build something that exactly reproduces the watermark. 1080i-capable HDMI capture systems are expensive but not 5-figures (or even 4-figures) expensive nowadays, and most satellite feeds don't even justify 1080i anyway. Do satellite receivers do HDCP? If they do, RGB video is still very very very good quality for 1080i, and reference-quality RGB capture systems aren't 5 figures (probably very low 4 figures). You could just output via HDMI or whatever of course.

To complete the setup, you could have one place receive a legitimate broadcast, and (alongside the TVs/projectors) run that through a capture system which would do differential frame analysis to heuristically figure out what watermark is displaying at the moment. This location automatically pushes the current watermark type out to a bunch of other places - for a nice small fee.

Of course, the above is said from an honest academic-security standpoint. :P
If you'd like a taster of morse, I can highly recommend the Android and iOS app "Morse Toad". It teaches the basics of morse code in a gamified format.

If you're serious about using morse code, I'd suggest the LCWO trainer. It's a browser-based suite of tools for learning morse. The UI is a bit clunky, but it does the job.

https://play.google.com/store/apps/details?id=com.mikelovesr...

https://itunes.apple.com/gb/app/morse-toad/id906586079?mt=8

https://lcwo.net/
I don't know what you do at Spotify but I have seen some of your company pitch books and your two paragraph analysis is one of the best succinct descriptions of the company I have read anywhere. Hope you're in management, if not you should be. Good luck.
Facebook is comparable to opium in the 18-19th century. Dictated by powerful organization, some considered it harmful but most considered it normal, addictive in nature, and people consuming it wastes precious time. In no way I am saying facebook equals opium, just pointing out some aspects of it is potentially dangerous to our generation (So as many other stuff, e.g tobacco, alcohol, etc).
So far to my knowledge only Winamp is able to open a dir with a lot of mp3's and subdirs in a blink of an eye and have all the functionality, including search/jump available. Others (foobar, XMMS) fail badly. I killed the foobar2000 task after waiting for several minutes.
Making a billion dollars a year revenue is pretty much the definition of "benefiting everyone", except when you're a part of regulatory capture (which AirBnB and Uber are not... yet).
A decade is 10 years. 1995 to 2015 is 20 years. So in terms of time elapsed it's 2 decades.
shiftwidth, tabstop and expandtab
Is there really value in raising up an ideal that doesn't hold in practice in any way shape or form? It seems as if our interviewing methodologies are designed simply to have a guise of meritocracy whereas in other fields that guise is dropped entirely and we see what really exists for what it is (mostly bullshit).
I'd argue that REAPER is the Vim or Emacs of DAWs. There's a steep learning curve, getting the best out of it requires a lot of memorisation and customisation, but nothing matches its speed and power in the hands of an expert.
If you wanted a discussion, you wouldn't be throwing out a one line bomb. Since you didn't raise an objection to anything I said, I can only assume you agree with my characterization. As they say, "It ain't slander if it's true."
This one might've used a vertical-blank interrupt to time the refreshes -- it's what I think I remember, but I could easily be mixing it up with some other article around then, like Chris Crawford's series on Atari 800 graphics.
Two things:

1. This is overly simplistic. What happens if your IO and logic are by necessity interleaved? Grab X out of DB, grab Y or Z out of DB depending on X's value, etc.? The whole thing just reeks of "ideal case".

2. This is overly complex. All that really needs to be said here is "pull out your pure code when possible". There's nothing special about F# to enable that. The logic in "tryAcceptComposition" is just a function calling other functions; you can do that in C# or even C. The only advantage F# adds here is the piping syntax, which to me only serves to make the code more obtuse. But I guess you couldn't write a three-part series about a single "extract pure function" op.

(This brings up an interesting thought: ReSharper should come up with a way to let you highlight a function and extract the "obviously pure" tidbits automatically).
What is a good hosted ipsec vpn? also is there built in support for networkmanager?
any way to import existing data from ledger?
Higher quality?

I always feel like that they avoid special effects and CGI on purpose in the Netflix series.

It is way to obvious that they want to save money there. Even if it means that the story (line) and world building looks and feels wrong.
cinst licecap
I once checked this out and as it turns out there's quite a lot of cases where foreign aid is spent on weapons, even sometimes as a condition of foreign aid.

I was was stupefied that Belgium made a foreign aid donation to Saudi Arabia. I found the reason : it was contingent on the Saudis spending about 30x the amount of the donation on weapons made by a plant south of Brussels, with (I'm sure coincidentally) several (then) cabinet members on the board. They bought "weapons for crowd control" with it, because apparently that's what fully automatic machine guns are used for.

I think the issue is that weapons production is an industry that is entirely controlled by government action, both in the positive and negative direction.

Governments forbid certain parts of the market on one hand, and they cheat and lie and deceive everyone so their own weapons manufacturers would see more customers on the other hand. US, Israel, France, even tiny little Belgium all engage in this.
You mean, just like the original list promotes further punishment of people who are victims of baseless arrests that don't even support prosecution much less conviction?
Question to Libertarians... Is this statism? Or legitimate penalties for externalities? Kinda hard for all those parents of kids with asthma to sue all those drivers and manufacturers after the fact. Isn't this reported result a benefit that wouldn't have arisen in a purely anarcho capitalist society?
Thanks for your post! I remember reading this sometime in 2013 and writing my own minesweeper AI based on it. It didn't get very far, but I sure had a lot of fun.
Ask HN: What is your favorite text editor?
I am new to learning python. Some people recommend to use IDE or others to use text editor. What do you say? What should I use?
Arrested and then jailed for possession of Meth in 2006. He would have been 51!
Setting Cookies in React – Netlify
As a local, you probably are more interested in keeping your networth, at least that's what most people want. I am not sure if they would necessarily want their home prices go down.
Looks like an erd from databases.

I think I didn't see one since like ten years?
What is linear math?

This stuff is taught in intro discrete mathematics courses.
I think that's about as close as you can get to a usable metric.

Of course, it still has problems. The metric probably has to be calculated long before the number of serious bugs is even known. On the other hand, good developers anticipate needs; there may be features implemented that the users haven't even realized they wanted yet. And of course, it's hard to come up with numerical weights for the features and bugs.

But the worst problem with this metric is that it doesn't count the maintainability of the code. That's an even harder thing to measure, of course.
1. Speed of automated tests. 2. Often it is good to test the semantics of a sub system are solid before integrating it so that edge cases are routed out and at the very least it's easy to find where a break occurred.
At one point licecap stopped working for me (MPB, using an external screen). The Giphy capture tool is also a good one. I wholeheartedly agree that including screenshots are a great way to show things :)
Calculating what it would take to park a solar sail at Alpha Centauri
VLC is great, but UI sux tbh.
Also, SMplayer is great, but UI sux even more for audio.

The level of 'it just works' for those 2 products is just not comparable to anything else. Plus they are cross platform, works on pretty much anything in existence.

That is the reason I tollerate UI and use them every day. Also, while it lacks audio management, SMPlayer is the best video player, period.
Especially in winter, after you get sick once with a cough? Doctor took one look at me, and told me to take a Zantac twice a day for three days.

Cough was gone. It was miraculous!
I think that many people don't understand that clinicians are not investigators and generally do not have any research experience, despite having the title of 'doctor' (a fairly recent appellation outside of academia).

They are, as you say, trained to arrive at a diagnosis to the best of their ability and provide treatment. A challenging diagnosis, however, is a fully-fledged scientific investigation. This is where clinical researchers (MD PhD) can really shine, but it's hard to apply this to individuals; the typical direction follows funding first, which may not be of much help to any given patient. At any rate, this approach ultimately does not scale.

My hope is that new tools (accessible databases, better data collection, and a dash of ML) will get more patients in front of the right pair of eyes more often. The situation with life-threatening diseases is bad enough, but there are also huge opportunities for improved quality of life from vague, nagging, or intermittent illness diagnosis.
Tunable eyeglass lenses to replace bifocals
See my Twitter discussion with the post author, it touches on this https://twitter.com/yawaramin/status/827963406988742656
I now use https://github.com/junegunn/vim-plug - you can setup a one-liner in your Vimrc to download and set it up, then :PlugInstall and you're done (similar to what they have in the readme but I linked to a specific hash on GitHub instead of master).

I also keep a separate file that lists the plugins to install and the source that from my main .vimrc. A lot of my coworkers have wanted my main vimrc for an "out of the box setup" of Vim and found it best for them to specify their own plugins rather than just blindly use mine - it also makes it easier for them to grab the latest copy without any conflicts.
You may find it interesting to go over a similar conversation I had with the post author a few days ago https://github.com/ploeh/ploeh.github.com/commit/f89c405e92b...
What's wrong with the normal screenshare on hangouts?

My company uses that fairly extensively with no issues.
I transferred some stuff over from Spotify to Apple Music last week using SongShift[0] without any issues, but YMMV.

[0]: https://appsto.re/us/MmCCbb.i
Thats a tough choice, even for people with years of experience.

Programming is not just writing code, its a `tedious` task of writing, compiling, running and debugging. IMHO, as a started you need an IDE to get things done fast (at least to get the environment up and running)

After that you can easily switch to a better editor like Vim(again personal choice)

Also check this article: http://softwareengineering.stackexchange.com/q/20950
I read the first three comments and they all say roughly the same thing. But I can't figure why you think this functions as you hypothesize. Do you have data?

Why not move the buttons on HN to random parts of the page? Or randomize the article titles every time you reload HN, so you're never sure whether you've already read an article? Or swap one comment between threads to keep people on their toes? Wouldn't those all be equally plausible ways to get people to read things more carefully by sacrificing usability?

Your other comments suggest that you would be against tl;dr's, which outside the internet are known as "abstracts", and (like descriptive titles) are known to be highly useful.
Microsoft Password Guidance[pdf]
Sea Level Regulated Tetrapod Diversity Dynamics
Who Isn’t Profiting Off the Backs of Researchers?
The digital community was lacking, but I spent time in Costa Rica and found it very safe, inviting, and beautiful.

I was in a very small surf town on the Pacific coast with a strong ex-pat presence.
Academic Throwdown Between John Preskill and Frank Wilczek
I'm in love with Rethinking Innateness. I'm posting half in the hope someone suggests an up to date version, but it's brilliant and fascinating itself. It was prescribed when I was studying my CS MSc and I've read it three times now.

I got initially into computers because I would press something on the keyboard and knew there was a sequence of commands that directed the graphics card to output something – I needed to understand that. Now, as I assume applies to so many people here, my curiosity has expanded to human development, this books is _the bridge_ between CS and psychology.

https://www.amazon.com/Rethinking-Innateness-Connectionist-P...
I'm not sure I follow.

I didn't explain my point well now that I've re-read it but the idea is more than the early maverick hacker projects felt more like just a project, something that was made to make sure everyone's lives got better. Uber, AirBnB, et. al., were business from the beginning. Their interests look to be making money, not providing a service or making it better. The focus isn't on making transportation or hospitality better, it's about controlling a market with their system and earning a profit. Motive matters, and it just doesn't strike that they are really benefitting anyone but themselves.

This isn't saying their services can't be useful - heck, I have an AirBnB reservation for my holiday the next 3 weeks. But I also don't doubt that if AirBnB would change their service in an instant if it suited them. They're only doing hospitality because there's money in it. That's not really the same spirit and it's not really with our benefit in mind, it's with our wallets in mind.
(OT) I discovered goldfrapp by watching Hard Candy. Awesome movie.
Fraud on my Credit Card and my experience
I don't think they're just listing the top 3 conferences. It seems they're listing the conferences that are considered "top" tier conferences - e.g. venues that would "count" for graduation as a PhD student / consideration as faculty, at a top-10 CS uni.
I'm curious what background you have in ML. "The difficulty in machine learning was always in the noisy and unstructured datasets like images" is truly a baffling statement to make.
Same boat here.

Vim + tmux definitely helps with managing loads of things.

Add w3m to the mix.
I am certainly no libertarian but I think the framing is a bit uncharitable, anarcho-capitalism is a radical fringe, the vast majority of ideological libertarians (my estimate) are okay with the state internalizing externalities, although there would be objections to its ability to do this fairly and effectively