skysight-inc/HackerNews huggingface dataset:
The article makes many good points but also misses on a few in my opinion. Hadoop was designed to solve a pretty narrow problem a long time ago; modern use cases tend to be far outside its intended design envelope.

Being more explicitly Linux-centric and dropping the JVM paves the way for some major performance optimizations. In practice, virtually everyone uses Linux for these kinds of applications so portability is not a significant concern. Efficiency in the data center is becoming a major concern, both in terms of CapEx and OpEx, and the basic Hadoop stack can be exceedingly suboptimal in this regard. Free software is not cost effective if you require 10-100x the hardware (realistic) of a more optimal implementation.

However, I would use a different architecture generally rather than reimplementing the Hadoop one. The Achille's Heel of Hadoop (and Spark) is its modularity. Throughput for database-like workloads, which matches a lot of modern big data applications, is completely dependent on tightly scheduling execution, network, and storage operations without any hand-offs to other subsystems. If the architecture requires network and storage I/O to be scheduled by other processes, performance falls off a cliff for well-understood reasons (see also: mmap()-ed databases). People know how to design I/O intensive data processing engines, it was just outside the original architectural case for Hadoop. We can do better.

A properly designed server kernel running on cheap, modern hardware and a 10 GbE network can do the following concurrently on the same data model:

- true stream/event processing on the ingest path at wire speed (none of that "fast batch" nonsense)

- drive that ingest stream all the way through indexing and disk storage with the data fully online

- execute hundreds of concurrent ad hoc queries on that live data model

These are not intrinsically separate systems, it is just that we've traditionally designed data platforms to be single purpose. However, it does require a single scheduler across the complete set of operations required in order to deliver that workload.

From a software engineering standpoint it is inconvenient that good database kernels are essentially monolithic and incredibly dense but it is unavoidable if performance matters.
> Surely the percentage varies significantly depending on the drug.

It actually doesn't, which is one of the key findings of addiction research. The reason is that people don't abuse drugs because of the physical discomfort of withdrawal, but rather because they have a psychological dependence on drug use. And that psychological dependence is caused by things like previous life trauma; any given drug is just an interchangeable object that's being fit into a slot.

The one drug-related variable that is important though is route of administration. Because addiction involves linking an action with a reward, the faster a drug takes effect the more likely a user will form a problem with that substance. So basically in terms of addictiveness:

smoking > mainlining > snorting > eating > topical

This is why different drugs seem like they have different addictiveness potentials.
For what it's worth, I upvoted despite my disagreement about IBM being supposedly complicit in South African institutionalized racism (I don't think the EFF's case here is particularly convincing as presented in the article, though the case regarding Nazi collaboration might be different), since I think your comment does in fact contribute positively to the discussion and doesn't deserve to be downvoted at all, let alone as heavily as it seems to be.
No conspiracy. They were going to use it for something else.

http://webcache.googleusercontent.com/search?q=cache%3AjPUYS...
What's an 'Elsefie'? Here, let me explain...
Being average looking should land you the most responses: http://blog.okcupid.com/index.php/page/14/
And lkml discussion of the follow-up patches: http://lkml.iu.edu/hypermail/linux/kernel/1502.1/00694.html
Have you tried rereading any book that people consider an eternal classic?

A math textbook typically wouldn't fit the bill (though certain exceptional textbooks would of course) and Madiq Alley gets good reviews, but opinion seems mixed.

Hamlet, by contrast, is still widely read centuries after publishing. I expect Wodehouse will continue to be read long after we're dead (plus the author of the original piece had a personal connection).

There's no guarantee you'll like rereading of course. But I've found I can't reread most books, but others I can read many, many times.
Once you see it this way, it's hard to stand in line at Starbucks and think "I'm not an addict." Everyone and their mother is addicted to something, it's just a matter of whether or not the authorities approve of your choice of substance.

Truth is we're humans and humans seek to alter their consciousness whether it be through meditation, prayer, exercise, spinning in circles or a through a shortcut -- a substance.

Unfortunately we're working against decades of misinformation and racism in this fight for our freedom to consume substances as we wish. It's unfortunate because I think there's a lot of medical value in a lot of the substances our government has deemed illegal.
> One does not even need jurisdiction in order to try someone for war crimes.

Yes, one does.

> The Kuala Lampur War Crimes Commission tried and convicted George W. Bush and Tony Blair for "crimes against peace" in 2011 under the principle of universal jurisdiction.

Ignoring the question of the legitimacy of the KLWCC (as others have addressed that issue effectively), the principle of universal jurisdiction is a basis under which to assert jurisdiction, so its use is an acknowledgement of the need to establish jurisdiction, rather than support for the claim that jurisdiction is unnecessary for such a trial.
While I am responsible for moderating the [nodejs] Google group, I'm not the only moderator, and have been too busy to spend much time on it recently. It's not my intention to squelch discussion, and I'll touch bases with the other moderator to make sure that we're at least being consistent.

At the same time, one of the primary reasons for moderating the list in the first place was to ensure that the group is constructive and useful, and not just a venue for flamewars. This has been a problem when discussions around foundations, Joyent, forks, and the like have happened in the past. In fact, it nearly killed the group last time this discussion happened (a lot of longtime community members with a lot to contribute gave up on it in disgust), so I'm not going to claim that we're not going to step in if things get overly heated.
> Yet the only people I've known who had problems with methamphetamine or heroine have all ended up hitting rock-bottom or dying.

Have you known anyone who hasn't had a problem with either of those drugs but still used them at some point in their life?

I would recommend reading TFA, the statistic isn't something the author made up, its from a leaked copy of a United Nations Office on Drug Control report.

For those interested in the statistic:

"Only 10 percent of drug users have a problem with their substance. Some 90 percent of people who use a drug—the overwhelming majority—are not harmed by it."
What's worse is that they won't let this be disabled by an about-config switch. They're basically turning Firefox into an iPhone and AMO into the App Store, amputating the add-on ecosystem and users’ add-on experience without any consideration for what end users actually want.
I'd say it's pretty risky either way, but you could insulate yourself from risk somewhat by using a raspberry pi with a fresh image on the SD card each time.

Also, if it's possible to run a raspberry pi from a write-protected SD card (and assuming that SD-card write-protection hardware switches are actual hardware disables not something that just sets a flag that the software can ignore), then you may not need to clean the SD card every time either.
> but there are a lot of things we can no longer do as a society because our public domain is no longer growing.

Serious question: what useful advance is being held back by a bunch of abandoned entertainment not going into the public domain?
Judge rules for NSA in warrantless search case
> trying to see if I could implement `docker build` using available commands

This should be possible soon! The only command missing is a symmetric `docker cp`. I've got most of the implementation ready to be reviewed in a pull request [1] (closed now, but I will reopen it soon).

[1] https://github.com/docker/docker/pull/10198#issuecomment-733...
White House to Create New Division to Streamline Cyberthreat Intelligence
Sure, but it was the only primary source I could find for Drew Houston recounting his conversation with Steve Jobs. Most all the other coverage at the time was blogspam.
we use all the sensors on the device, including compass and accelerometer. This is only possible because we built our own end-to-end mapstack for mobile.
This silence on FOIA requests is in line with what the FBI has been instructing local law enforcement to do[1]. Here's the relevant FBI letter:

In the event that the Minnesota Bureau of Criminal Apprehension receives a
request pursuant to the Freedom of Information Act (5 USC 552) or an equivalent
state or local law, the civil or criminal discovery process, or other judicial,
legislative, or administrative process, to disclose information concerning the
Harris Corporation [REDACTED] the Minnesota Bureau of Criminal Apprehension will
immediately notify the FBI of any such request telephonically and in writing in
order to allow sufficient time for the FBI to seek to prevent disclosure through
appropriate channels.

This is a written policy that FOIA requests should by default be resisted. This combined with the change from a mission of "law enforcement" to "national security"[2] signifies a huge shift in the FBI away from a rule-of-law culture to a rule-of-man culture.

[1]: http://arstechnica.com/tech-policy/2015/02/fbi-really-doesnt...

[2]: http://www.msnbc.com/the-last-word/fbis-main-mission-now-not....
Here's an interesting small extension of the problem: prove that it is not possible to guarantee survival if 64 is replaced by some number that is not a power of 2.
I agree that IBM should be punished for violating American law. My point is that the focus of the EFF's brief ought to be specific cases where IBM actually did facilitate Apartheid (if they did, in fact, do so). Creating a citizen database (for example) is not immoral, nor is recording race as statistical information, but adding a race identifier to the ID numbers does seem to be, and thus would be a better case. "IBM is bad because it implemented a racial identification code in the second-to-last digit of citizen ID numbers" is a much more convincing case than "IBM is bad because it created a citizen ID numbering system".
Product/Founder Fit
Hi, co-founder

This is something we probably should have spoken directly to in the article because of how popular spark is. The main reason we didn't is that we like Spark and don't feel it needs to be replaced. It doesn't seem to have most of the ecosystem problems we discuss because it's got a company behind it. My understanding of Spark is that it's designed to be used with different storage backends and I'm very curious to see what would happen if we got it talking to pfs. I think it could work very well because Spark's notion of immutable RDDs seems very similar to the way pfs handles snapshots.

Hope this clears a few things up and apologizes if I've mischaracterized Spark here I've only used it a little bit.
How about not existing?

Seriously, this was originally 0.5. However, because people had started to use 0.4.6 as a pseudo-1.0 in production, and because the internals are completely different, it's a little bit of version jumping to a base we actually want to call our stable branch.
VZW is hard-throttling unlimited data plans on 4G LTE
A hint of explanation of the reverse-engineering:

https://www.quora.com/Many-articles-about-Drew-Houston-say-t...

Update: I haven't found video or slides about Rian Hunter's explanation, but I did find this note about his CUSEC 2012 talk:

At the Dropbox talk, Rian Hunter explained how Dropbox was able to integrate into the Mac OS X finder and the hoops that they had to jump through to implement this feature. The technical aspects of the talk were centered on how the engineers used very low level C code to trick the operating system into thinking that the Dropbox client was a part of the finder when it actually was not. An unexpected but enjoyable surprise was the presence of an Apple engineer who worked on the OS X operating system. It was almost if trying to break the software was a game of cat and mouse between Apple and Dropbox, but in the end, both Rian and the engineer created an extremely informative and enjoyable discussion.

-- http://iwarrior.uwaterloo.ca/2012/02/01/13721/softies-invade...
So I'm building a yellow-page type web app
Can I populate the DB with a bunch of local businesses myself or do I need to ask them? Is this legal? Essentially it's free advertising.
Enjoy it while you still can :). My children were the same way. Now they are making jokes at my pronunciations and grammars. [English is not my first language.]
I'd love to apply to YC, but I cannot under any circumstances move to the bay area. :(
In analytics it pays to know where your data is from – Provenance is everything
BrowserSync 2.0
Here is the actual ruling itself:

https://ia801403.us.archive.org/22/items/gov.uscourts.cand.2...

Excerpt:

> Disclosure of this classified information would risk informing adversaries of the specific nature and operational details of the Upstream collection process and the scope of the NSA’s participation in the program. Notwithstanding the unauthorized public disclosures made in the recent past and the Government’s subsequent releases of previously classified information about certain NSA intelligence gathering activities since 2013, the Court notes that substantial details about the challenged program remain classified. The question of whether Plaintiffs have standing and the substantive issue of whether there are Fourth Amendment violations cannot be litigated without impinging on that heightened security classification. Because a fair and full adjudication of the Government Defendants’ defenses would require harmful disclosures of national security information that is protected by the state secrets privilege, the Court must exclude such evidence from the case.

So the court isn't ruling on whether NSA surveillance is constitutional. They're saying NSA surveillance is too secret for them to be able to rule on, because state secrets privilege.
> That means the explanations are to be found in genetics and evolution, not psychochemistry.

Right, the point here is that perhaps the psychoactive substances aided our ancestors in making decisions or enlightening them(or straight up enhancing their senses) to withstand the test of evolution.

I'm not saying it gave them superpowers, but perhaps it enabled them to look at a particular challenge in a different perspective.
The only thing wrong with this comment (IMO) is the assertion that companies help governments kill journalists. I'm not asserting they don't, but I don't consider it common knowledge that they do, i.e. please provide a source.

I upvoted.
You are right that devicemapper (basically lvm snapshots for storage, used as an alternative to aufs/btrfs on many distros including Red Hat) has been a source of headaches for us. It is relatively obscure (we knew we were in trouble when googling libdevmapper error messages returned our own source code as the first result) and frankly not pleasant to work with.

But, the good news is that we have made progress recently. In fact the 1.5 release includes several improvements to devicemapper. And as of last week, Red Hat has volunteered a dedicated engineer to escalate devicemapper problems. So, fingers crossed things will be even better in the next release :)

Another alternative is to switch storage drivers: aufs and btrfs are popular options.
Provenance is an increasingly popular topic in the food industry as quality scandals and ethical sourcing of produce push their way to their way to the front pages of the press but have you considered how tracking the provenance of your data can add value to your data analytics?
that second one [edit] isn't a con. it doesn't even (reading literally) imply that the hours are bad, just that they start at 10 or 11/noon. It doesn't make your point at all, but just says schedule is flexible, with a bias toward working late. great.
Or I think he means you should be spending more on R&D and marketing.
I would assume that it's possible, especially with a Linux laptop, to prepare a USB port to simply mount a drive and not do any of the other magic (like installing human interface devices, or whatever) that leads to vulnerabilities.
Thanks for following up Solomon! I'm glad you are at least aware of what is going on with this.
Here's one of your assertions I challenge:

if IBM didn't do it, another company would have done it.

How do you know that? How can you prove that? Honestly, I wouldn't blame someone for stopping to read at that point.

Here's another thing you said:

IBM is a company, an entity. it is made of people, but
it is not a person [...] but a company does not have
morals.

If (a) IBM is a company made up of people, and (b) people have morality, on what basis are you claiming IBM doesn't (read: can't) have morals? A collection of people acting on behalf of a corporation does not absolve that corporation from any moral corruption it causes.

You speak about this as if you've never read any of the comprehensive literature on business ethics. I think you're being downvoted because you're coming off arrogant in a topic (business ethics) you don't project knowing much about.
I lived there for 4 years and was aware of "tree town" but never heard anyone call it that. I know it as A2.
An earlier take on it:

https://ocfnash.wordpress.com/2009/10/31/yet-another-prisone...
Node v0.12 only passes tests because of this: https://github.com/joyent/node/pull/8689

io.js does not contain this commit, or any of the changes from this commit: https://github.com/iojs/io.js/search?utf8=%E2%9C%93&q=%22PAS...
This is a good analogy if the debt is invested in your personal financial growth somehow, for instance through education or perhaps actual investments, but a terrible analogy for the kind of deficit-spending most people do, like on cars and high living expenses.
> for anyone who wants to write isomorphic code without a lot of pain in compilers, is limited by browser adoption

Are you confused? "isomorphic" doesn't mean what you think it means ...
This dude got it right one this one wow
How the Nathan Barley Nightmare Came True
The JVM runs every language I care about, in a mature system with a simple, stable interface. It has its problems (startup time), but for the kind of jobs you use Hadoop for it's rarely an issue.

Switching to Docker feels like a real step backwards. Rather than a jar containing strictly-specified bytecode, cluster jobs are going to be random linux binaries that could do any combination of system calls. You need another layer of build/management system to make your docker containers useable. Worst of all, rather than defining common shared services for e.g. job scheduling, this is going to encourage users to put any random component in their docker image. We'll end up with a more fragmented ecosystem, not less.
From their front page:

> io.js is an npm compatible platform originally based on node.js™.

Sorry, they were aiming for 100% tests passing [1], I would assume for npm compatibility which is their primary goal.

A stable API means that dependent packages don't have to change to stay compatible. Tests are failing because they change things that broke packages:

* Node-inspector does not work with io.js! How is anyone supposed to debug their code? [2]

* Fibers do not work [3]

* msgpack does not work [4]

* sqlite3... there is a long list [5]

There is no CI, no debugger, tests aren't passing, major modules don't work. I absolutely cannot go to my CEO and make a bet on that for my company.

I think io.js is great! But the amount of misinformation about the state of Node is staggering. Let anyone with proof otherwise also cite their sources.

[1] http://blog.izs.me/post/104685388058/io-js

[2] https://github.com/node-inspector/node-inspector/issues/523

[3] https://github.com/laverdet/node-fibers/issues/203

[4] https://github.com/msgpack/msgpack-node/issues/20

[5] https://github.com/iojs/io.js/issues/456
Shop Amazon – TurboTax – Get 10% More on Top of Your Federal Refund – Whamdeal
Can you point out to what exactly is notable about this code? I don't see anything special.
I note that this is written by Johann Hari http://en.wikipedia.org/wiki/Johann_Hari
One of the more obvious ways in which drugs could have played a part in our evolution is, roughly stated, how the consumption of certain substances may have kept those individuals with more developed brains from going mad, or counteracted some of the disadvantages, allowing the trait to be passed on.
> But we need a better option on the web in order to achieve the kinds of experiences that people have come to expect from native applications.

I think this is a really cool project and a technical achievement, but I disagree with this one statement. We don't actually need to "achieve the kinds of experiences". This is my biggest problem with designers I've encountered, they do not see the benefits of the platform and try to design everything the same way. The web has distinct advantages that makes it special and I think way too much effort is put into making it behave like native applications (trying to catch up to its benefits rather than leaning on the web's).
Show HN: Operation Go: A Routine Mission
Operation Go is an online programming game for Golang. Developers play as one of two agents on a mission to defeat the notorious cyber-criminal Epoch.

It was a ton of fun to make. I hope you enjoy it too.

Happy coding :)
Spot Is Boston Dynamics' Nimble New Quadruped Robot
EFF Vows to Continue Fight Against Mass Surveillance After Disappointing Ruling
Should be tagged (2012)
rustic, rusty turtles, even
sounds less irritating
9N 89W is off the coast of Costa Rica, 9S 91E is off the coast of Singapore, which both locations seem to be kind of close to areas prone to earthquakes and tsunamis.

Also, it says it could have changed half a billion years ago. The current estimate of the formation of the moon is around 4.5 billion years ago, maybe these are related and they got their dating wrong?

The only thing I can find that happened half a billion years ago was the end of the the Neoproterozoic Era (the end of the Proterozoic Eon), which saw the first multi-celled life and the end of the Marinoan galciation, which could have subsided through global release of methane from equatorial permafrost.
Tinder robot reveals men are not as terrible as you might think
/dev/gibson: Because you need more Hackers in your kernel