skysight-inc/HackerNews huggingface dataset:
> calling external APIs probably isn't something you want to be doing in Hadoop anyway

And yet I've seen it done. At least they weren't truly external, just external to Hadoop.

> In the former case, we're taking rapid steps towards YARN anyways. I'm sure non-Hadoop, pre-packaged systems exist that can handle this, although I'm not personally familiar with them

There's those goggles again :) Batch schedulers have existed for decades (e.g. PBS started in '91).

> (such as smooth scaling without having to think about hardware),

This neatly embodies what I believe is the primary fallacy in most of the decision making, including the fact that it's often parenthetical (i.e. a throway assumption).

Does anyone really value the "smoothness" of scaling? I'd expect the important feature to be, instead, that the slope of the curve doesn't decrease too fast.

The notion that Hadoop someone frees one from having to think about hardware flies in the face of hardware sizing advice from, Cloudera, Hortonworks, and others that discuss sizing node resources based on workload (mostly i/o and ram) expectations and heterogenous clusters. It does, however, explain my observation, in the wild of clusters built out of nodes that seem undersized in all respects for any workload.

>It's now really easy to spin up a cluster,

It's really easy if it's already there? That borders on the tautological. Or are you talking about an additional cluster in an organization where there already is one?

>and with small scale, costs are not that big a deal.

That's just too broad a generalization, just as "cost is a big deal" would be. Cost is always a factor, just not always the biggest one.

Small scale is often (though not always) associated with limited funds. Doubling, or even tacking on 50% to, the cost could be catastrophic to a seed startup or a scientific researcher.

>With large scale, your system is distributed in some way anyway.

This strikes me as little more than a version of the slippery slope fallacy. Even some web or app servers behind a load balancers could be considered distributed [1], but that doesn't make them a good candidate for anything that's actually considered a distribute framework.

It also hand-waves away the problem that, even if costs weren't a big deal at small scale, they don't somehow magically become less of an issue at large scale. Paying a 50% "tax" on $40k is one thing. At $400k, you could have hired someone to think about hardware for a year, instead.

[1] I just recently pointed out, slightly tongue-in-cheeck, an architectural similarity, one layer down, between app server and database https://news.ycombinator.com/item?id=17521817
There are a lot of YIMBY organizations and similar things popping up these days. It's a good way to get involved without doing everything yourself.
Bitcoin is not monetarily inflationary. (and definitely not price inflationary, long term). Its limit is known well ahead of time. The distribution is spread over time, by design. But the number of bitcoins is mathematically capped.
That was my problem with the OP's research. It presumed that techies differed from fuzzies because only techies like tech. Bzzzt. In fact, many techies have very broad interests, including fuzzy fare and math and science. Stratifying your population using a flawed premise is a bad start toward producing results that usefully describe more than a niche group or two.

Then to presume that an article on black holes which is complex and challenging would cause those with an interest in them to lose interest... That conclusion is just silly. The right conclusion is that these people now have less interest in reading complex and challenging articles... perhaps on any topic.

In all, I don't hold much hope that the results of this research will replicate well.
Google Maps recently started using landmarks. Yesterday it told me to turn right after the IHOP.
I don’t think that explanation makes sense. It isn’t about regular computing power, it is about the hash rate of the mining hardware. They use specialized hardware (ASICs), and if you look on this page you can see a breakdown of the hashrates:

https://en.bitcoin.it/wiki/Mining_hardware_comparison

The top one (other than the Dragonmint miners which are not listed) is the Antminer S9 which you can see was released over two years ago.

If you are aware of some new mining hardware that is much faster please let me know.
It is worth remembering that the skills and the interests need to be combined in your life, not every day at work. Maybe you are passionate about gardening so you have a good job that pays well and keeps summer hours, letting you spend your weekends gardening.

I have a passion that I engage with for a living (math) and it's currently keeping me underpaid. I'm also very interested in art and so I go to museums, work on learning how to paint, etc -- and if it was my job, I'd stop liking it. I like it so much because it's a complement.

Some passions are like hot sauce -- a sprinkling makes almost every food better, while a full bowl gives you the runs.
"How much time, energy, and capital is appropriate to expend on this situation?"

That's none of my concern. I honestly do not care about how much trouble this kind of thing is for AirBnB and other such companies.
Ray Lewis (NFL linebacker) said something along the lines of "I get paid for Monday through Saturday, Sunday I do for free".

I feel like that with coding a lot of the time. I get paid to deal with all the crap work, but those times I get to code something novel or unique are times I would still want even if I didn't have to work.
This kind of proves the point. You're misunderstanding the mathematics. Asymmetry of friendship is irrelevant. Whether the friendships have non-local dependency is irrelevant. The kids' friendship and non-friendship IS a colored complete graph: between any two nodes (kids), there is either a red edge (if those two are friends--in the context of the group, if you want), or there is a blue edge (if those two are not friends--in the context of the group, if you want).
I decided, after burning out of Startupville a bit ago, that I was going to take some time off and... wait for it... write a novel. I love writing, I love blogging, so how hard could this be? It's the next natural step, right?

I thought this would be a dream experience, sitting at my computer and creating beautiful prose for hours a day, my creativity flowing like a river.

After about 20,000 words (a debut novel should be between 70-100k words), it turned into hard work. One thousand words a day becomes quite tedious, especially when the initial steam wears off.

Great, so I pushed through for months, putting in the daily grind. Draft complete at 90k words! First round of edits, cutting almost 20,000 words (about a month of work, all for "nothing"). Back at the writing grind again to get the word count and story aligned to be more marketable.

Edit number two complete. Edit number three complete. Beta readers telling me there are big holes here, there, everywhere, This character sucks, this character doesn't make sense, etc etc etc. Just now wrapping up edit number four, and I'm actually quite happy with how it reads.

But now... onto the business end of this deal, querying agents. Agents reject about 99% of the pitches that come into their inbox. The 1% gets a full manuscript request. Of those 1%, about 10% get offers of representation.

An offer of representation often turns into several more rounds of edits before the agent will sell it to a publisher. So if I'm lucky enough to get an agent, it'll be even more editing (my least favorite part of this process so far).

THEN, the agent has to successfully sell the book to a publisher. The odds that this happens also pretty low. And since I'm a first-time writer, I'd be looking at a pretty small five figure advance for what will end up being about two years of work.

Anyways... writing, my favorite form of art, has become a roll-of-the-dice grind. I'm not sure I would go back in time and do this again, but regardless, I've come this far so...

I think chasing a "dream career" whatever that is can be a tough lesson in setting realistic expectations in what it means to make money.
Anyone being able to take the photo and you being recorded all the time are two really different thing.
Because the scaling effects of automation change the natural balance of the implications of no public privacy.

For instance, the South Dakota supreme court recently found that leaving a webcam on public property for months in order to record everyone who showed up to and left a private residence to be a fourth amendment violation.

https://www.criminallegalnews.org/news/2017/nov/16/south-dak...
Stop posting shit from this site, please.
> U.S. invested about $200 million

> China spent $101 Billion in quantum physics in 2015.

Wow that's pretty staggering
Learn the Fundamentals of Webpack v4 with Core Maintainer Sean Larkin
Lol, releasing Clintons dirty laundry is somehow "hacking an election"
Is anyone arguing in favour of open offices? I thought they existed because they were cheaper and then management tries to justify them after the fact by saying they foster collaboration.

Aren't these blog posts just preaching to the choir?
Ethereum would like a word.
Yes, only Russian agents are indicted this time, but someone who might be Roger Stone makes a brief appearance (#44) in the document, too, probably to be targeted in future legal action.
IMHO, this is a great insight and result!
At least some of that seems in the past. Presently, Schindler has good coverage of the details here [1], which details what a generous reading of mkempe's point is trying to say -- let's not go over-board.

1. http://observer.com/2018/07/putin-conflict-poland-ukraine-na...
This is a terrible attitude. Deciding that companies should have free reign to do whatever has had disastrous consequences in the past, and today would be no different.
Same for license plate recognition. It is as if the privacy-deranged never thought about why your car has a gigantic retroreflective number plate on the front and rear. It is so that you car can be recognized and identified from long distances.
Oh really? Specifically, which one? Give me a link. Keep in mind the latest inflation reading today was 3.33%y/y. This is also an optimistic reading -- if you use the exact same calculation the fed used back in the 80s inflation would be at 10% (many government obligations such as TIPS are linked to CPI and, as such, they are motivated to make that number appear low).

Risk free savings used to be much easier when the dollar couldn't be freely printed. Sure, there were times of moderate deflation and moderate inflation, but overall the average was 0. Furthermore, labor was able to consistently achieve wage gains. Those days ended in the 70s (and is evidence by the famous wage-vs-productivity charts). Also, you're conveniently ignoring the fact that fiat currency has made it possible for governments to print money to pay for war instead of needing to levy a tax on its citizens (something which would frequently be met with protest).
I'm not sure I understand the grammar of the HN title. What does "goes in length" mean?
IMO HPMOR is a rich reading experience with nothing besides familiarity with Harry Potter as a cultural phenomenon (i.e. you know that they are wizards and they go to wizard school). It really picks up about 60 chapters in though, and that's a lot to ask someone to get through.
> I've seen how regulation really can work for consumer benefit. I'm not sure if people appreciate how much COPPA cleaned up in child privacy

Yes absolutely! Similarly, HIPAA has been incredibly useful in forcing hospitals and other for-profit companies to at least kind of care about medical data security. It's not perfect, but without HIPAA that data privacy/security would be in a far worse state.
Smart TVs are invading privacy and should be investigated, senators say
I'm going to do my best to have a good faith argument. I agree with the assertion that the US elections should be free from foreign influence, however, Russia is accused of simply bringing unfavorable things to light, it doesn't seem like they invented/constructed anything, they just amplified prexisting tensions - they're accused of organizing anti-Trump rallies and espousing pro-BLM views, certainly that's unacceptable influencing but it's not a dynamic because of the Russians.

But the Russians weren't the only ones who did this, the President of Mexico was firmly against Trump and I recognize that as an unacceptable foreign influence. How do you recognize the difference between acceptable and unacceptable foreign influence and was the President of Mexico acting in a way that was acceptable?
Actual article title: "What it takes to stand up to authority"
Hi Comrade!
agreed. happily typing on my 2013 now. best laptop in existence. and it cost half as much.
Bitcoin is a deflationary currency.
I think they mean 'in detail'
Open offices are a success because they achieve their goal of reducing costs. The "collaboration" was wishful thinking that didn't pan out.
If I had to hazard a guess, it would be for more than US MSRP in order to incentivize the seller.
There was a meeting with a foreign ambassador offering to collaborate which was met with enthusiasm by trumps people who we are to believe knew nothing about this even though he was in the same building a few floors above.

He then went on TV and called for the Russians to hack Hillary.
I think you're focusing on TOML's weakest case here. I think for a lot of configuration files, TOML is going to be easier to read and write. Personally, I'd rather have a configuration file that looks like this:

[site]
name = "My Great Website"
url = "https://example.com"
author = "Watts Martin"
email = "foo@bar.com"
links = [
{ name = "tom", url = "http://tom.example.com" },
{ name = "bob", url = "http://bob.example.com" }
]

[database]
server = "localhost"
username = "dbuser"
password = "dbpassword"

Than one that looks like this:

{
"site": {
"name": "My Great Website",
"url": "https://example.com",
"author": "Watts Martin",
"email": "foo@bar.com",
"links": [
{ "name": "tom", "url": "http://tom.example.com" },
{ "name": "bob", "url": "http://bob.example.com" }
]
},
"database": {
"server": "localhost",
"username": "dbuser",
"password": "dbpassword"
}
}

Semantically, I just find the first one clearer and more intuitive. (The links are really only the dubious part.)

The equivalent YAML doesn't look bad:

site:
name: My Great Website
url: 'https://example.com'
author: Watts Martin
email: foo@bar.com
links:
- name: tom
url: 'http://tom.example.com'
- name: bob
url: 'http://bob.example.com'
database:
server: localhost
username: dbuser
password: dbpassword

...but the significant whitespace makes it somewhat more fragile, particularly in those pesky links.
No, but I'm certain that whether revered or not, Braudel's work is still "full of mistakes and bad reasoning" -- it's just that in history those are harder to tell.
Been an early beta user of outclip for a few months now. Robust app, works well and looks very polished. I like that the devs have started to focus on the bug reporting use case. Looking forward to seeing where they take this!
They hacked her email server and released her dirty laundry. Not exactly "hacking an election." I'd call it whistleblowing.
>>Seriously. I don't want to live in a world where I am "stalked" by billion dollar "people" who decide whether I get insurance, and at what price, and who can search "big data" for any and every possible "infraction", also retroactively, at their "discretion".

If this can be automated passively and cheaply, its an inevitable outcome; regulation or not.
Is it possible...and I'm just conjecturing here, that this advice is for the 0.01% of humanity. Think about it from an extremely high level POV. If giving this advice to everyone in the entire world, and if 0.01% is able to follow through with it...this is how we get standouts like Beethoven, Einstein, Gates, Jobs. etc. Is this terrible advice then? Sure, it doesn't help plebs like us, but it sure does help the world move along...
Cool, was thinking about this subject earlier today.

Abstract:

>This paper addresses the scalability challenge of architecture search by formulating
the task in a differentiable manner. Unlike conventional approaches of applying evolution
or reinforcement learning over a discrete and non-differentiable search space,
our method is based on the continuous relaxation of the architecture representation,
allowing efficient search of the architecture using gradient descent. Extensive experiments
on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our
algorithm excels in discovering high-performance convolutional architectures for
image classification and recurrent architectures for language modeling, while being
orders of magnitude faster than state-of-the-art non-differentiable techniques.

Zoph's paper on architecture search: https://arxiv.org/abs/1611.01578

Concept of architecture search from ages ago: https://static1.squarespace.com/static/58e2a71bf7e0ab3ba886c...
Facebook scooping up Instagram has to be one of the best corporate acquisitions over the past 20 years. Hindsight is 20/20 but it makes sense that a simpler social network like Snapchat and Instagram would be a threat to Facebook once everyone's grandma got an account. A really interesting counterfactual would be a world were Facebook, Instagram & Snapchat are all competing.
Though poverty does cause ruin, the cause of poverty is often a lack of education and I'm not talking academic, I'm talking about the education your parents give you. Europe has been through a lot more bad experiences and doesn't want to do that again whilst America is yet to learn.
A politician isn’t supposed to enact their own will. In theory all you need to do is convince the constituents or the people who pay the politician with a public media campaign.
votes, things to hide, etc all change the page and prevent serving everything wholesale out of cache

I'm not actually sure, just guessing...
>>Laws are meaningless if no one follows them.

>Laws are also meaningless if no one enforces them.

And still, Rule of Law requires that those who enforce the law, follow the law.
Tesla Plots Bold China Factory Move, Stays Silent on Cost
Yes I noticed that too. I'm starting to think psychology may end up ahead of other fields actually - the replication crisis is much wider than just psychology but it seems to have come to the surface there first. And so they're the first to tackle it.
If I left bad rating to carpooling drivers who drive unsafely... I’d do it every time. For some reason, although I’m ok with one offense at a time, drivers really want to both phone and drive over the speed limit, and tail the next guy, at the same time, and the 4 witnesses in the car don’t worry them...
"On the evening of June 15, 2016 between 4:19PM and 4:56PM, defendants used their Moscow-based server to search for a series of English words and phrases that later appeared in Guccifer 2.0’s first blog post..."

That has got to be one of the scarier things I've read in some time as to the depth and breadth of current surveillance.
Yes, it was. Guccifer leaked documents which have been demonstrated to have been altered.
Life (and keeping up with the latest research) :)
Second layer solutions would like to speak to Ethereum about bloatware.
Lol the unions are essentially organized crime syndicates and political machines. They couldn't give two fucks about this kind of thing, and the ACLU is in sjw mode. Good luck.
You realize a video recording is just a series of photos?

How can you allow one and not allow the other?

How are you going to write that law? No more video recording outside? A 30 minute limit? No more documentaries?
thanks for the advice - yes I'm on the JVM w/ spring boot.

I suppose I may be overcomplicating things with trying to do it "the right way" right out of the gate. I guess the "perfect is the enemy of done" saying comes into play here.
It seems like the article highlights the non-traditional things Japanese whisky makers are doing (e.g. using woods other than oak in their barrels).

In other words, the Japanese whisky makers are successfully innovating and exercising more creativity than most American and European whisky makers. And it seems to be paying off in terms of flavor.
> doing your passion for a living can take the fun out of it

I think this is really a corrupt idea that floats around, and many times influences younger people away from trying to find a career and instead landing themselves in crappy jobs.

For example, some people like to work on cars, but they don't get a job in that because their dumb uncle said this to them, so they spend their time grinding away at a fast food joint and going nowhere in life. Maybe they can poke at their 15 year old car once in a while, but overall their quality of life is not very good, and they basically never get to work on cars. If they had instead gone into professional car service, they would have a decent paying job and be able to reach a comparatively higher standard.

Does that mean that it's not work? Of course not. You will always have to do work and it's not always pleasant, but at least you can leverage your interests and strengths. If you get bored or annoyed at working on something you used to think was fun, then take a vacation or find something else do to over the weekends to get your mind off it. If you are annoyed that it's not fun anymore, then -- welcome to adulthood.

That is truly the difference between having a job and setting yourself up for a successful career.
Hypothetically, if a Russian agent became President of the USA, what would you expect him to do? Try to withdraw from as many international treaties as possible? Insult foreign heads of state and sabotage America's relationship with their allies? Refuse to enact sanctions against Russia?

I don't think Trump is a Russian agent, but I wouldn't call the idea crazy
If you're spending $1 of electricity to mine $1 of coins, why not just spend $1 to buy the coins outright?
> You learned how to build the kernel, git, the kernel style guide, signed-off-bys, checkpatch, get-maintainer and finally running git-email.

These are all pretty well documented tasks. It might not be 'one button simple' like a gitlab merge request, but I thought it was pretty easy overall. It did take some time investment on my part to learn the submission steps (about a weekend total, including the compiling and testing locally).

> Large and/or controversial changes ...

I suspect if they are large or controversial, that in itself implies the barrier to merging is going to be higher. How many new contributors (non-paid contributors especially) want to merge something controversial on the first go?

Could it be simpler? Possibly. Sometimes complex systems do require complex processes for maintenance. I'm just saying I didn't think it was especially burdensome to contribute a small change as a brand-new, completely-unassisted by another person (before sending patch) contributor.
I think you may be missing the point -- the one being named and maybe shamed in this post is actually Airbnb.
The isomorphism is: let the kids be nodes. Between kids x and y, place a red edge if x and y are friends, or place a blue edge if x and y are not friends.
Addendum to this: HIPAA told us (engineers who work in the health space) what to do, and HITECH told us how to do it.

Both are useful.
Meetings between high level Trump campaign officials and Russia are public knowledge.

https://en.wikipedia.org/wiki/Trump_campaign%E2%80%93Russian...