skysight-inc/HackerNews huggingface dataset:
I think this could have been done with GNU parallel. One advantage I see with Spark is that is that it is easier to interact with Python, for example these two lines are all is needed to call the relevant Python function:

urls = sc.parallelize(batched_data)
labelled_images = urls.flatMap(apply_batch)

So if you already have a cluster with Spark installed (like Databrick does) then it takes less work to just call your Python code than setting up a GNU Parallel cluster and a writing a small wrapper script. Additionally a Python script would have to load/init the models on every call from Parallel. I agree that this is not a great demonstration of Spark main strengths.
On the contrary, how can he be bored if he's constantly roaming his imagination 20yrs ahead of where we are now, just as vividly as he was in 1994 writing about cryptocurrency, laptops made from straw, and reconfigurable chairs governed by topological expressions.
it says getting access to this service will get me access to photoshop too. That's a good deal as the service is just $10 per month.

but what confuses me is, on photoshop.com, it is still $20 per month.
Is it worth reading if I haven't read Flash Boys?

(I'm planning to read both, so I'm mostly just curious).
Check out the Swarm Intelligence's prediction on the election: http://unanimous.ai/goodperson/

Very interesting way to view public sentiment.
I remember using this program in 2001:

http://linux.die.net/man/1/ttv

aalib + video4linux = TV tuner outputting as ASCII.

[Shockingly I actually used this for a useful purpose - I had a messed up graphics card that couldn't run X stably due to machine lockups, but Linux console worked fine - so I used this to watch TV. I was a teenager at the time so it was a while before I replaced it.]
Mind explaining who this is, and why this is an issue?

(I genuinely have no idea, and I figure you can probably explain better than a Google search could.)
How to generate intermediate files with clang to read Ruby's MRI implementation
So, I'm a nerd, and I found "Not So Far" more interesting than Lewis's book. I don't think you need to read Lewis' book to get value out of the rebuttal.
Building a product or how I'm moving from a job to a career
Thanks for this, I'd been typing way too much for too long. However, dc overlaps with the dc program so I've moved to dkm and dkc (since I actually use dc from time to time.)
> I happen to be involved with cryptography. I'm very familiar with three of the people on the Zcash team, one of whom I call "friend".

Maybe this is where your potential bias stems from. I mean, what's the point of name dropping here if it's not stemming from a bit of ego, maybe strong enough ego to believe you're out of reach for a three letter agency? You know three of them, great, but you don't know all of them and you may not know the most major contributor and it's possible there exists a key member that has an already shady past.

The hypothetical in question isn't even intended for you. It's intended for the general public. Most of whom would gladly eat up whatever story they are sold.
Deployment pipeline types you're not running but should be
I find it very confusing with flat node_modules as it doesn't hide complexity.
I've lived all over the world and I suspect it's a class thing. Downtown Chicago people tend to talk over you nonstop with no regard for courtesy. Where I've been in France and the rest of Europe, it's more what you'd expect (a pause). In my rural midwestern working class background there tends to be a pause, we didn't talk over one another. My wife's family is from Guadalajara, were privately schooled and in the Mexican government's "political class", they don't interrupt.

It's ingrained in me enough that if someone talks over me, I'll just stop talking to them and subconsciously consider them to have a low standard of manners.
You don't have to assume zero sum in general. Just that some interactions can be zero or negative sum. The simple example is monopoly rents: monopolists can increase their share of total profits without increasing value delivered.

By a lot of measures, the financial industry captures a lot more money than they did in previous decades. But it's far from clear that they're delivering a lot more value, and given that computers have drastically reduced their costs, it's reasonable to expect them to be smaller, not larger. As a start, this has some good graphs:

http://esoltas.blogspot.com/2013/02/5-more-graphs-on-finance...

And here are a couple more general-audience articles:

http://blogs.reuters.com/felix-salmon/2011/03/30/chart-of-th...

http://blogs.wsj.com/economics/2011/12/10/number-of-the-week...
Unfortunately it seems that Xhyve cannot connect to the Internet when traffic is routed through OpenVPN. Otherwise I would use it at my job. I tried https://github.com/coreos/coreos-xhyve, https://github.com/ailispaw/docker-root-xhyve and Veertu, and they all won't work.

Apparently is a known issue https://github.com/mist64/xhyve/issues/84 ; it might have to do with Hypervisor.framework
They would do it anyway, it just makes it more news worthy given the patents. Although, I'm not sure its zero sum depending on how it shakes out.
Giant Bomb have been dabbling with this in their "Film & 40s" podcasts (more for entertainments sake than being well-informed though).

For anyone with a GB Premium account, links are:

Rocky IV - http://www.giantbomb.com/podcasts/film-40s-rocky-iv/1600-115...

Hunt for Red October - http://www.giantbomb.com/podcasts/film-and-40s-the-hunt-for-...
That would require a complete overhaul of the social order... Unlike AI, however, nobody seems to be making any headway on that.
I don't know why the comment is being downvoted. There is a bunch of people that want what you want, it's just that, as you are saying, is a bit of a pipe dream right now. The way technology is progressing, it might be feasible in a few years.
Should I use Vagrant or Docker for creating an isolated environment?
I've tried it during the transition from Google Code. It works well enough but it didn't handle orphaned branches (as used by Github pages) well and I had to provide credentials twice for every push for some reason.
How about TypeScript/TSX?
React's components and "views" are easy to type (and you get completion in your IDE) but I still have to find a convenient way to type an Immutable map where each key/value pair has a different type (the equivalent of a TypeScript object). Any idea?
Dymatize Iso 100 hydrolyzed whey protein isolate 726g – 1.6lbs
There's an interesting property of fully untraceable currency, and that's that it's possible to perform 'perfect crimes' [1]. Often the weakness in committing, say, a kidnapping or a drug sale is handling the associated money obtained with it which can be traced back to people due to circumstances. Fully untraceable currency by definition completely avoids these circumstances.

The inevitable question "should we want this?" is moot because evidently such systems are possible (iff they provide a practical currency for such transactions) and will inevitably be used for this. Furthermore, Bitcoin already achieves something close to it in practice. It's just worth pointing out that untraceability will have nasty consequences as well as good ones.

As an interesting aside, David Chaum, who has lately been infamous for advocating ways to backdoor encryption, may be considered the father of fully untraceable (but not decentralized) digital currencies. Ironically, after [1] was published others in the cryptographic community spend a number of papers on building what they called 'fair' blind signatures, which constituted basically of varieties of Chaum's basic idea but where untraceability could be lifted by an "independent party posessing the right private key" (i.e. a public "police backdoor"). They were trying to backdoor Chaum's encryption...

[1]: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.465...
Aren't Angels, Seed focused VCs and incubators essentially patrons? In many ways the modern patron of the arts is the Angel investor funding technological arts that may never become viable businesses.
Is there a reason why javascript can't track if the page is on the front or in the back?
I believe millennials have only recently overtaken baby boomers as the larger voting demographic.

http://www.pewresearch.org/fact-tank/2015/01/16/this-year-mi...

The article also seems to miss the student loan problem and how education costs are no longer paid by working a summer job and often take ten years into a career just to pay off.
The Conversation We Need to Have Instead of Talking About a UBI
TL/DR
iOS is crap cause it is not customizable and because some of iPhone specs are getting worst (while still better on average).
So he search and found the holy grail of specs and install a stripped down alternative version of Android overly customized to remove feature from UI. And this is supposed to be simple and make your brain cringe less...
Land Rover Defender has been sold for 550,000 euros – APPlite
A lot of the flight trackers out there are using data from setups very like this! They are often crowdsourced (augmenting that by buying data for areas with little or no coverage like the FAA data for Atlantic/Pacific crossings).

It's a fun thing to setup and watch. :-)

We sell a similar receiver in our store! https://shop.pimoroni.com/products/mini-digital-tv-stick
Please do. My interaction with support has been laughably bad. I have a Dropcam that sends me a monthly warning that my account that's paid up for a year is due. I also have a Nest Protect that can no longer see my network because I have two APs broadcasting the same SSID. For the first one, customer service told me "ignore the emails". For the second one, customer service told me "permanently remove one of your access points"

Both responses are enough that I no longer recommend the Nest ecosystem for anyone. They clearly don't care about support, which is troubling for a company that's in the services business.
Show HN: Platform to discuss movies and tv shows
So the cool thing here is that you can use Spark and TF to find the best model like Microsoft Research did with Resnets.

http://www.wired.com/2016/01/microsoft-neural-net-shows-deep...

They're showing you how to train different architectures simultaneously, and then compare their results in order to select the best one. That's great as far as it goes.

The drawback is that with this schema, you can't actually train a given network faster, which is what you want to do with Spark. What is the role of a distributed run-time in training artificial neural networks? It's easy. NNs are computationally intensive, so you want to share the work over many machines.

Spark can help you orchestrate that through data parallelism, parameter averaging and iterative reduce, which we do with Deeplearning4j.

http://deeplearning4j.org/spark
https://github.com/deeplearning4j/dl4j-spark-cdh5-examples

Data parallelism is an approach Google uses to train neural networks on tons of data quickly. The idea is that you shard your data to a lot of equivalent models, have each of the models train on a separate machine, and then average their parameters. That works, it's fast, and it's how Spark can help you do deep learning better.
This is why we have verbal cues for conversation that indicate that the listener-now-speaker has understood what was said and is formulating a reply: "hmmm, I see. But have you considered, perhaps, that..."

Conversation, as opposed to written text, has a lot of "filler" designed to give the conversant time to formulate their words.

Your implication is that people who respond "on time" have thought less about their response than those who do not. More likely, they have just a better intuition for conversation that allows it appear that way.
Still this is all just Ad-Hominem attack on the characters in Flash Boys and seems a bit over the top "Yeah, I might believe that. I might believe that if I were a dribbling retard."

doesn't really answer why Flash Boys does such a terrible job of explaining HFT
There is an app for that, its called Angie's list.
Containers don't run on osx, but docker isn't responsible for the containers anyway - lxc is.

The "docker" program builds and runs natively under osx just fine, like any other Go binary.
I know there was auto-industry resistance to moving away from sealed-beam headlamps.

In general I think it's widely agreed that all businesses almost universally tend to oppose most forms of gov't imposed change.
The creators themselves are apparently rather confident that it will indeed count, at least for the guiness record. The World Cube Association doesn't have a "robot category" as far as I am aware.

https://www.reddit.com/r/Cubers/comments/42ank6/worlds_faste...
With a special!?! With a special???!!
Well, NPM is changing to a flat structure in V3.

Complexity isn't much of a concern since all dependencies should (ideally) me automatically managed by NPM.

The greater benefit is reduced duplication and reducing excessive folder nesting. The former saves space and the time required to download dependencies, the latter solves some serious issues with using NPM on windows.
Oy gevalt! That again? It's the modern version of counting angels on a pin.

Uber said, "drivers get this percent from the fee, Uber gets this much". Uber claimed "tip's included". Modern philosophers claimed this makes a difference because the law prohibits taking a cut of tips.

But there is no difference! Any allocation of the fee can be rephrased as being "with" or "without' a tip in such a way that the money flows are all the same. There is no "fact of the matter" as to what part of the fee is a tip! Any phrasing can be correct!

Let's say Uber took 25%. You can say that the fee breaks down as:

1) 25% to Uber, 50% driver payment, 25% driver tip

2) 25% to Uber, 75% driver payment, no tip

3) 25% driver payment, 75% tip to driver, which Uber takes a 1/3 cut off (illegal).

If the law treats observationally equivalent situations differently because of how they're labeled, that law is meaningless, and basically just taxing you on angel pinheads. (I call the property of passing this test "nominal invariance".)

(The exception of course is when a customer gives a cash tip on top of the fee directly to the driver. But Uber definitely doesn't take a cut of that!
Reflection and Reification
I'm a perfect use case for this service, so I guess I'll tell you my story. My credit score is pretty low. Low enough that I apply for high interest cards with my bank and get rejected.

I have a high income and an equally high number of expenses. Each month I have about 300-600$ in "discretionary" income. I also have a partner with a very low income and a son. My partner is trying to get on her feet right now and there is a high degree in variability with her income and needs. She has recurring medical bills and there are constant one time medical expenditures when you have a child. Not to mention birthdays, once in a lifetime kind of events where your child gets to participate, there are family and friends that are worse off financially and need help from time to time.

I don't like, respect, or particularly want to use my banking services. I don't trust them but having a checking account is a requirement. Every time I have 500$ it makes me sick to my stomach to imagine handing it over to them, given the fees and backhanded tactics they use. I don't trust them. I imagine I might get a secured credit card sometime, but handing over the 500$ and waiting two weeks to a month for my own money back, again, just makes me kind of sick.

This service would help me in an emergency, and help me build credit (if not to the same extent as a secured credit card)
A couple of technical notes:

> such as a repository structure optimized for performance

Actually, repository structure hasn't fundamentally changed (unless you count the RevlogNG change in 0.9). There have been changes so that data on disk compresses better (the generaldelta change) and there's more underway to support new features (such as narrow clones), but they generally aren't performance-related.

I note that this is a good thing: Mercurial's repository structure allows it to do things that aren't easy for Git, especially efficient random access to history (Git's structures are generally optimized for access close to HEAD and the root of the directory tree), at the expense of requiring more disk space.

The performance improvements you've seen most likely come from more/better caching or reimplementation of time-critical code.

> fast branches without making a separate working directory for each

That was always possible. Documentation (such as the Mercurial book) suggested a clone-based workflow for reasons of convenience and because that's what people were used to at the time, not because you had to do it this way.
Same goes for Cisco AnyConnect VPN. Updating the routing table seems to temporarily fix it, but it would be nice not to hack around it to get it to work.
For those curious about the speech Lewis gave at Princeton referenced in the article, here it is:

https://www.youtube.com/watch?v=CiQ_T5C3hIM

https://www.princeton.edu/main/news/archive/S33/87/54K53/

It's one of the most memorable speeches I've ever heard or read. A personal favorite.
...or just get one in a colo for $40/month.
My father is in his 60's. He literally cannot type. EMR laws have increased his day by 2 hours. In regards to scribes... It's PHI which is dealing with HIPAA, having medical training to understand terminology, etc. Hiring someone to understand "Arthroplasty, acetabular and proximal femoral prosthetic replacement" is expensive. That was just a hip-replacement, imagine dealing with something like cancer. I realize he's somewhat of an exception, but people that spend their day talking to patients rarely get the amount of typing practice that people who have the time to browse the internet do.

In regards to the second question, that's what I do. If you've ever worked in any corporate setting that has industry-specific software, convincing the company to change it is near impossible. "Time saving" is incredibly difficult to demonstrate and "cost saving" can take 2 years to show anything meaningful. Software for a complicated field ends up being complicated no matter how hard you try to not be the java-megastructure catch-all application, which means it will always take some amount of training. This means you have to convince someone who just lost 2 hours of their day to spend more time learning a new application. Welcome to the shit-show of corporate software.

As my father loves to counsel: "It's hard to get the nine-to-fivers to listen to the five-to-niners".
Old people: "They sure don't make 'em like they used to!"

Me: Thank goodness.
I've watched this story unfold more and more as a friend was laid off by Disney and not rehired. Money talks in these situations, without a public backlash against Disney it'll be cheaper for them to keep doing this.
Another movie I'd love to see something like this for is Fast Times at Ridgemont High. There exists a TV-friendly cut that removes all nudity, and inserts a couple of extra scenes (like Spicoli in the boys' room bragging to his stoner buds about what he's going to do to Mr. Hand). I'd love an edit that starts with the R-rated release, and adds in the extra TV-friendly footage.
It's an interesting article. There's certainly a lot of untapped talent out there doing relatively menial work when they've shown capability to do very different and more effective things. Sponsoring such people or just being more likely to hire them could bring benefits. Well, it's already been proven by the companies doing it while reaping those benefits. Plus the occasional academic team asking for funding on something that might be a game-changer and is to some degree. So, we need more of it.

That said, the author is wrong about the title and doesn't get Silicon Valley. Most don't because they don't know the historical context of how it became what it is. If you want to re-create or beat it, you have to either imitate or improve on the components that went into creating it. This is the best article I've seen summarizing the big picture of how Silicon Valley came to be:

https://medium.com/backchannel/why-silicon-valley-will-conti...

Note: I bet many people didn't think Zuckerberg and others paid visits to the old guard to learn old lessons for IT. Probably thought it was all modern, fresh thinking "disrupting" the old stuff. A mix as usual. ;)
I feel like this isn't much simpler than the official docker-machine workflow, but I may be too far in the weeds at this point to objectively tell how complicated it is. In my case I need to test tools across 4 versions of Docker so I'm locked into using docker-machine and dvm.
you can use the functional way of setState by passing a transaction function (state: State) => State . This allows for typed, immutable updates since the function gets passed a copy of the state.

If you need an object (say to use it in something like Redux), you have to write a wrapper like so: https://gist.github.com/ebi/d186c4297c84d562aeae
Yeah, sure, I agree, there's a lot of room for improvement. But, I still talk to a lot of developers who are new to Docker and use OSX, who are confused as to why they need to run a VM or enter docker commands into a different terminal than they are used to.
Given Amazon's history to NEVER involve law enforcement in outrageous cases like https://news.ycombinator.com/item?id=10966164 or even the OP, I doubt you would get arrested.
Yup, the key is to take back control and make the phone for your convenience. Rather than the convenience of others and apps.

Limiting social media accounts is helpful in this too. I have FB and Twitter but receive no emails or notifications from either and don't even have any FB-related apps on my phone at all. They sit there waiting for me to visit at my convenience. I check FB maybe once every 3 months at my desk. When I do look, I always realize I'm missing out on nothing.

If people need to reach me, I check my email a few times a day on my phone/desktop and can get immediate access through my cellphone number if they have it.
> To my surprise it came not just unlocked but rooted.

This rings alarm bells for me. What are the chances that it comes with a pre-installed Android rootkit? How would you know?
I would suggest reading both at the same time. The rebuttal is organized to follow along with the format of the book just for this purpose.
I think that is a fair criticism. You can read my blog post to be "Flash Boys, Not So Fast is the rebuttal I would have written if I had the time and was a good writer".

Maybe I should change the post to be more explicit about that.
The caste system has left its mark on Indians’ genomes
If onre's comment doesn't suffice, your picture is the backside of a P1800ES. It was released in the early 70s. Before it, during the 60s, Volvo had started using 3-point seatbelts (front and back), padded dashboards, rear-facing child seats, crumple zones and head restraints.
Can anyone elucidate why having artificial barriers to employing certain types of people is beneficial?

If so, why not create the same restrictions on, say, a company in California from hiring someone from New York?

Or is this a guise for xenophobia?
You don't take into account tooling. There're more options for tools which work with git than for hg. Also, git support is usually more stable and better tested.
This is one of those cases that neatly separates the (ivory-tower) academic types from the (getting-hands-dirty) engineer types of people.

People who say that drilling the wholes in the cube diminishes the achievement, the perfect-or-nothing crowd, belong to the first group.

People who see the drilling of holes as a valid tradeoff to achieve higher performance are more pragmatic and belong to the second group.