skysight-inc/HackerNews huggingface dataset:
I installed the previous release (luna) a while ago on an older pc I had laying around. I was really impressed. Though much of it is a ripoff of OS X design it's a pretty well done ripoff and coming from a Mac I think it's the most usable linux distro I've used so far. I'd love to see more adoption of this project, as it has a lot of potential to make linux more usable to non-technical types.
Docker with OverlayFS first impressions
This is riddled with problems.

a) I routinely load 6GB of financial data into R in less than 45 seconds, and it's complex and hierarchical to boot. Admittedly that's in R's native file format but even a flat CSV takes less than 5 minutes.

b) Why SQL? Has this person considered Pytables or h5py? Here is an excellent file format created by guys who work with large data sets all day long, which is vastly superior to an on-disk relational database for data sets of this size (and much bigger). Moreover, they map directly to Numpy.

c) Why SQL (II)? Why is all the wrangling being done in slow SQL when Pandas has 100x faster in memory querying capability via groupby and query methods?

d) The visualization is terrible. "Other" dominates the dataset and dwarfs any of the more interesting categories and their relationship to time of day. This is a very clear case for when a chart grid would have been preferable, easily constructable in Pandas or of course GG Plot, and surely doable with Plot.ly. We would have been table to compare how different categories have different time-of-day profiles.

e) If we're going to push data visualization tutorials into the big wide world, why not introduce people to quantile plots, useful here to compare the distribution of complaints (literally one line in R), or more simply and very intuitively, we could have superimposed all the hour-by-hour complaint density plots on each other and see how they compare (again, literally a few lines in R as in plot(density(x)); par(new = TRUE); plot(density(y), add = TRUE, col = "xxx"). After having normalised the data of course. This has its analogues in Python. A second, bog-standard, unstacked, bar chart could then have been used for aggregate complaint volume by hour, and dare I say it, even a pie chart would have been better here for comparing aggregate complaint category sizes.

This looks like a pitch for Plotly aimed at casual HTML developers, not serious data scientiests.
> In my experience using grunt, gulp and broccoli as task runner / build systems for complex applications, they all end up doing about the same thing.

This may be true for a codebase of trivial size. With a sizable codebase and many build steps (defeaturefy, babel, es6 modules, es3recast, jshint, jscs) you quickly see grunt and gulp fall flat.

Gulp and Grunt absolutely have a simpler API for blindly chaining tasks. And they have the ability (as task runners) to define multiple tasks and compose them.

Broccoli is tricky to learn. In Ember-CLI we go out of our way to make it invisible to devs who shouldn't need to care what tool is running the internals. For the internals however, it is definitely the correct tool.
I think that's terrible advice for learning Linux since most people learning Linux are hoping to learn to use the shell and the programs accessible only via command line.

I tried learning Linux since the mid-2000s and failed every time up until recently. I even followed your advice to the letter. I would install Ubuntu and just played with the default apps and tried downloading some other stuff but quickly got bored because I wasn't learning anything and I wasn't doing anything productive on it. Everything I learned I learned in the first day of two of usage. It was that simple. I would always end up booting back into Windows regularly and reclaiming the disk space when I needed it. Then I'd repeat the process a year later.

I finally actually started to get Linux around early 2013 when I decided to try learning again but this time I had something useful to use it for: programming. I've been programming on Windows and wanted to try building software on Linux. Now I had a need to learn my way around bash and all the utilities found within it. I've also taken Intro to Linux course when it was offered on edX last year and although I can't say it was the greatest course I've ever taken it definitely would have helped me out a ton when I started using Ubuntu for my programming.

In fact, it is the best resource I've come across for learning Linux a beginner can find simply because it gives you a good idea of what working with Linux is like for a variety of perspectives. It's got a bit talking about working with text editors (both gui and cli based), a bit talking about users and groups, chapters covering the basics super-useful utilities like grep that you really ought to know exists.

My favorite thing I learned from the course was the `apropos` utility which searches the man pages. One of the worst problems I've had with Linux was remembering the name of some program that did something useful. Now I can just type `apropos pdf` to find a brief description of all programs that have the word pdf in their man page description. Super useful when your searching for that pdf file splitter or merger you used a blue moon ago but can't remember its name.

In fact, I don't think I've ever bothered with man pages up until I've taken this course. Now I do a whole lot less Googling and more fending for myself.

Sorry, this went on for too long but I think my point is clear; you're probably not going to learn much if you just install Ubuntu and this course (in my opinion) has incredible value for a beginner.
A good summary of why probabilistic models are important is this article by the legendary Chris Bishop: http://research.microsoft.com/en-us/um/people/cmbishop/downl...
DOCKERCON CFP SUMMARY
The TLD's are not about searches, they are about that universal title that hangs in that text input constantly hovering over the top of all your content.
Definitely need more posts/articles like that.

Sometimes publicity is the most efficient way to eradicate vendor' sloppiness and ignorance.
Ask HN: Zerigo DDoS has taken my site down – other options?
My site is now down because of a DDoS attack on Zerigo. Does anyone else use them, and have people migrated to other services?

http://www.quill.org
How to Node.js EventEmitter
I just realized that the Amiga story is exactly what would have happened to Apple in an alternative timeline where the company had filed for bankruptcy without Jobs. Or the other way around, if they had succeeded we all may be wearing Amiga Watches on our wrists right now.

It's very interesting to see how good/bad managerial decisions or tiny details can totally sink an advanced technology and change the course of the future. Can we imagine a present with a current technology without the advances that, for example, Apple has brought? Maybe we'll all be still using Nokia phones or Palm PDAs.
awesome read, can't wait to see your results on performance, etc.
Let's make a small amendment to your simple statement.

>The simple explanation is: people use Javascript because it's a good language. That's the big factor in Javascript's success.

>The simple explanation is: people use PHP because it's a good language. That's the big factor in PHP's success.

Doesn't feel right, does it?

Google and corporate sponsors are incredibly important in building the ecosystem of programming language. Traction may be the single major defining characteristic of language success and one can certainly buy traction with money and influence.
That's just bad, this is breaking API's ... suddenly other logging deamons need to change their code because systemd is squatting the log file ?!?
I recently spoke to a reporter from one of three major newspaper in Israel. I asked her how much investigative journalism they do? She said almost none. The smallest of the 3 papers does a little. The other two do basically none. A little means a lot less than they do of everything else, of course.

So maybe 5% of the combined expenditure for the newspaper industry is probably a generous estimate. It could be <1% I guess.

Most of what journalism does is reporting, commentary, discussion and analysis. Primary, investigative journalism is a relatively small thing.
Willz – Drill
But you won't be sentenced to 300 years in prison in Denmark.
Apple's considerable open source contributions?
About MathWorld
Multithreaded ray tracing C# version: https://github.com/wieslawsoltes/rt
How do you maintain your privacy with a torrent-based network? Aren't there lots of trackers? Will we be able to encrypt our packets and send them as TCP so we can bypass ISPs?
> ...is it that unreasonable for them to say it is big data?

Let's say I have an organization and we run into a dataset that is 1,000 times larger than anything we've dealt with. Should we put out a help-wanted ad for a "big data" developer? What if the largest dataset we had previously dealt with was 100 rows? The reason we have terms for things like this is to facilitate communication. If the definition is highly sensitive to context, then the term doesn't facilitate any communication, the whole context must still be spelled out. If the term is to have any meaning at all, it can't be a relative thing. Of course I'm of the opinion that the term is already meaningless, so I guess do whatever you want :)
I think the issue is just stability in general. I just moved to a new server recently, and I decided to go with CentOS 6 instead of 7, as people were reporting that CentOS 7 is just a little less stable in general than 6. (Although systemd probably isn't entirely responsible for the lower stability in CentOS 7, it probably is one of the major changes in CentOS 7 so it likely takes a good share of the blame).

You always end up with bugs in released software, even CentOS. Some bugs only show up in certain situations, and might be difficult to track down. The only way to give a good guarantee of stability is for a large number of people to use the damn thing for years on end!
You broke HN ;)
Ask HN: Does artificial intelligence algorithms forget?
Humans forget things things and skills that we do not use for a while, maybe it's because we do not have infinite storage capacity in our brains and such a brain would use a lot of energy. Does artificial intelligence algorithms forget things they have learned but not used for a while? Does artificial intelligence algorithms have artificial dreams or the equivalent to redistribute their learnt memories?
10 top EdTech investors from around the world
Are they complementary? As in, is it worth to read the book after reading the blogposts, or do they tell the same story?
I think the problem is that the typical engineers that work on these devices are used to work on a whole different layer - hardware. But they know how to put together a few simple CGI scripts and probably don't even know the dangers they are facing when they run HTTP(s) server.

So on the one side you have HW guys saying "no need to find someone, we can do it ourselves" and on the other managers gladly accepting this on face value (it probably does cost them less in the short run). I am not saying these guys are not good engineers, just that this is not their area of expertize. Their mistake is not realizing this.
For the servers I run, boot time isn't an issue. I might reboot a server once a year, and booting in 10 seconds vs 40 seconds isn't really an issue. What is an issue for me is stability. I don't really care about "hideous shell scripts" as long as they work.
Uscaciunea vaginala-tratamente naturiste(vaginal dryness-natural treatments)
you mean as its default init system? I don't think so (unless you consider LFS a distribution), but it's available for many distributions plus it seems like a good init system for containers, etc.
Sheesh, I never got into angular because the "DI by parameter name" thing seriously bugged me but had no idea that the variable scoping in models could depend on the order that the inputs are used! How can people live with that?
Do you know if the Elementary OS Luna gets updated automatically to the new Freya version?
This. The minute I read "SOAP" I imagined the fault lies there. It probably does, but this was lower hanging fruit... :)
Aquamacs is a heavily customized emacs based on GNU Emacs. I'm not sure how or if Aquamacs resynchronizes with GNU Emacs, but in principle, a new release of Aquamacs might start with this release of GNU Emacs as a base.

In general, I've found the platform integrations of Aquamacs to not be worth the trouble and package incompatibilities that they appear to introduce. I'm not using OS X these days, but up until a couple of months ago, I tended to use either the "Emacs for OSX" distribution, or compile my own via Homebrew.

I think it's pretty common for people to recommend using one of the more "vanilla" versions of GNU Emacs over Aquamacs these days.
It's called native advertising and it's somewhat controversial. The problem is that, even if it's labeled, it erodes the wall between advertising and editorial. Many magazines have had "special advertising sections" forever, but the more you blend in native advertising, the more readers are likely to confuse your editorial with an advertiser's pitch.

Of course, especially in some areas, PR narratives do drive the choice of story (as suggested in your link) a lot of the time. But it's still a slippery slope to rent out column inches for content that purports to be a form of editorial.
I think the trick with systemd is to realise that it's not unix.

I keep coming back to that. It seems to me that there is a tension in the Linux world about people who want an open source UNIX and people who want an open source Windows. Systemd is much more a "Windows" approach to system configuration management than a "UNIX" approach.

That isn't "bad" per se but it makes for some confusing things. I would be ok if Linux became the open source windows flavored OS and Freebsd became the open source unix flavored OS. I believe such an explicit step would help folks move forward with "where" a certain idiom of OS management should land in the open source world. Of course I don't expect that to happen :-).
When the Echo came out, I created a proxy to add functionality (http://alexaho.me). One of those functionalities was a Hue integration, which Amazon integrated natively this week.

All of my code was open sourced. I never received any credit or compensation from Amazon for my idea or code, if they happened to look at it and frankly, I wouldn't try to fight them. Who would try to fight a company with such a large legal team?

Even a thank you for my idea would've been nice...
Postdoctoral Fellow/Scientist/Software Engineer in whole-cell modeling - Mt Sinai School of Medicine, New York NY

The Karr lab in the Institute for Genomics & Multiscale Biology at the Mount Sinai School of Medicine is seeking talented, ambitious individuals to develop cutting-edge dynamical models of human and bacterial cells. Despite the explosion of experimental data, we do not comprehensively understand how phenotype arises from genotype. Our approach is to develop computational models which predict how behavior emerges from the molecular level by integrating diverse data into a single model. Our goal is to use models to make medicine more precise and personalized by predicting disease progression and drug responses. We are also using our models to engineer bacteria. Our research is highly interdisciplinary involving computational modeling, software engineering, data visualization and curation. We have close collaborations with several experimental groups to build and test our models.

Responsibilities: The candidate will be responsible for developing human and bacterial whole-cell models for precision cancer medicine and synthetic biology. This will include expanding the scope of whole-cell models, incorporating new sources of high-throughput genomic data, and improving the accuracy of whole-cell models. The candidate will work closely with experimental cancer collaborators at Mount Sinai, as well as with Luis Serrano's laboratory at the Center for Regulatory Genomics in Barcelona, Spain.

Desired skills/experience: The ideal candidate has the following skills. Applicants with deep experience in related fields are also encouraged to apply.
- Deep experience in dynamical and genome-scale modeling, software engineering, data visualization, and databases;
- Strong knowledge of biology, physics, mathematics, and computation;
- Several years experience in computational systems biology is a plus;
- Excellent object-oriented programming skills in Python, MATLAB, and JavaScript;
- Ability to learn new skills and concepts;
- Excellent oral and written communication skills; and
- Abilities to work independently and within a team effort.

More information: Please visit our website (http://research.mssm.edu/karr/join.html) or contact Jonathan Karr (karr@mssm.edu).

How to apply: Please send a cover letter indicating your research and career goals, a CV, and 2-3 letters of reference to Jonathan Karr (karr@mssm.edu).
I won't argue about a particular phrase, as 1) buzz-words more often mis-used than used properly 2) there is no single, strict definition of it.

But going from ~MBs to ~GBs change methodology (of course, it's more about number of items and their structure than sheer size). Even at this scale there is some tradeoff of quality/quantity. And feasibility of using some more complex algorithms.

Data cleaning - sure, for all data. But when volume is in MB, it is still possible to open it it a text editor and manually look into abnormalities. Once it's GBs some artifacts will remain undetected.
The problem with file watching in JS land isn't the libraries, its node. I'll point you at the documentation [1]. fs.watch is inconsistent, unreliable, unperformant on certain platforms, and a layer too low for most applications. For that matter, npm is worlds better than python which is why you don't tend to see crazy dependency graphs in python.

[1] https://nodejs.org/api/fs.html#fs_caveats
A hundred billion more like.
A browser that's not Firefox or Chrome scares me slightly, considering the amount of resources that go into securing them, yet they all fall at cansecwest.

How vulnerable can I expect to be running Midori?
By the time Intel sold > 66 MHz 486's, the Pentium was already out. They remained a low end alternative, but still topped out at 100 MHz. AMD took their 486 clone (called the 5x86) eventually up to 150 MHz.
You misread the sentence. The comparison is being made to the Mesozoic. That is, the Mesozoic was much hotter than the Cretaceous. I can not imagine what you mean when you write "Global temperatures don't correlate ... with the amount of carbon sequestered in fossil fuels." I never suggested any such thing. But we do know that carbon can increase the temperature, and if we dig up an amount of carbon that was buried over 500 million years and we burn it all during a 200 or 300 year time span, then presumably we end up with more carbon in the air, at one time, than ever occurred before, and therefore we should expect the temperature to spike to a level never seen before.
I wonder how sustainable is their development method, are the UI changes to apps accepted upstream?
Is your comment saying 'linux is amazing, I just can't stop ricing it and get to work! OSX is better because it doesn't let me customize it'?
An interesting thing about probability is that it's essentially a "softened" version of logical reasoning, so maybe it's more fair to say that logic-based AI was generalized:

http://bayes.wustl.edu
disgusting. the government have been trying key escrow for three decades now. no. NO. _never_. we will never let it happen. over my dead body.
I like your idea. But sadly I am in no position to help you. I wish you good luck!!!
> start your own publishing company and give out the materials for free

There are many people that are moving exactly to this. As ever, the problem is that the market is not perfectly fluid and things like academic cultural biases, impressions of prestige, etc often do not follow the most efficient contours. Posts like the GP's are called for precisely because they call attention to this and try to shift perception.
If you build a custom logger into a startup/scheduler instead of using the existing logging component, you're doing it wrong.

Really, if you're not using the logging mechanism to check a system, what are you using?
Lufthansa customers were targeted by a cyber attack
Tech does exist everywhere and so do markets and YC has funded companies internationally and in other cities/states.

The only requirement is that you be in the bay area for the duration of the program which I think is essential because of the networking opportunities,office hours and dinners that you'll be attending in person.
What I gave up was any real hope at getting a job, this because I quite prominently link to two of my essays about my mental illness at the top of each page of my website: http://www.warplife.com/

"Living with Schizoaffective Disorder" comes across to most people as lucid, informative and helpful, but I regard it as rather pedestrian myself. It claims that "I want to tell you what it's like to live inside my head", but really, it doesn't.

What does tell you is "My Deepest Fear". I've been tossed involuntarily into psychiatric hospitals on three different occasions - twice by the police, in handcuffs - for no other reason than simply mentioning that essay to mental health professions. This despite not having been symptomatic in any way at the time I wrote it. That it reads like a paranoid manifesto was rather like the literary equivalent of a stage actor being "in character".

I'm pursuing some software projects that I have some real hope of succeeding with through crowdfunding, however in reality I regard hosting my software on my site, as a way to attract readers and inlinks to those two essays.

It is flatly illegal, in the United States, to discriminate against the mentally ill for reasons of employment. That doesn't make any difference to me; I still get no offers.

I could sue under the Americans with Disabilities Act however I feel very strongly in the assertion that "the best response to hate speech is more speech". Consider that had the Weimar Republic passed a law like the ADA, it would not have stopped Hitler from putting people like me to death.
Yep, that's what the whole Open Access movement is about, and it is making good progress.
>> The parts really did run 2x as fast or 4x as fast internally.

Actually, the Intel 486DX4 ran at only 3x the speed, despite the name. Intel couldn't use the DX3 name because of a trademark owned by AMD, who also had a part called the Am486DX4. The AMD part was also available with a 40MHz bus and a tripled 120MHz CPU clock.
Not knowing how to operate a server, I have to assume it's harder and you have to know everything it does and the CLI to use it. Not the same for mint or any other pre-packaged distros.
Scala.js: Safety and Sanity in the Wild West of the Web
> With their current API, we can't use the code written for RN on the web.

We actually designed React Native to work on the web. The layout and styles are a strict subset of CSS, the base component such as <View>, <Text> and <Image> can be easily implemented on-top of <div>, <span> and <img>. The gesture system is written in pure JavaScript as a React event plugin and should work as well.

It's just that it hasn't been a priority so far to exercise this possibility. But, every once in a while we make some example runs on the web. For example, two days ago during a hackathon we made 2048 example run (and work) in the browser by modifying less that 10 files.

> The end game isn’t ReactNative. We want the web to win.

Even though I was one of the few people that started React Native, I --really-- did not want it to exist in the first place. But, the sad truth is that you cannot build high quality, gesture heavy, apps on mobile web today :(
Ask HN: Big Data case studies
Anyone have good, specific examples or anecdotes of genuinely big data and the architectures you used to analyze it? (And/or counterexamples of 'big data'...)