skysight-inc/HackerNews huggingface dataset:
I wish that Facebook as new AOL would just have the same outcome and give way to a free and open internet. BUT Facebook gets to be AOL2 in a world where they have far far far more clout (order of magnitude larger user base and many orders of magnitude more significant impact on real life — not just an online diversion but central to politics, life, work, business, etc) and are smart enough to learn all the lessons from AOL's history in order to not repeat the same mistakes. Facebook's internet.org lost in India but won elsewhere. We're talking major power struggles where it's not clear that Facebook will lose in the end.

Google is indeed a whole other monster with a different set of issues. It's more of a blob with tons of parts, some more benign than others. Some parts comparable to Facebook, some more legitimate.
I'd be interested to see if something like what happened in computer backgammon occurs with AlphaGo in which long standing assumptions about how to play prove to be wrong.

TD-gammon, a computer backgammon player, explored "strategies that humans previously hadn't considered" [1] and led the backgammon playing community to re-evaluate some rules of thumb they used in opening moves [2].

[1] https://en.wikipedia.org/wiki/TD-Gammon

[2] http://www.bkgm.com/articles/tesauro/tdl.html
>allowed to benefit from logging and destroying their own primeval forests

What a sorry idea to still see afloat in 2017, that even the last remnants of special natural habitats should be allowed to be destroyed for "benefit"
I saw a pretty cool documentary on how reintroducing wolves to Yellowstone changed the ecosystem for the better.

Not the one I saw but very similar: https://www.youtube.com/watch?v=ysa5OBhXz-Q

More serious article:

http://www.yellowstonepark.com/wolf-reintroduction-changes-e...
Is it from Silicon Valley (tv show) ?
You are probably right.

But then you come across the 1% case and Kafka is the only thing that you can throw at the problem that works.

Not sure of exact numbers, but I think we're doing 4-5m writes per second across one Kafka cluster, and around 1m writes per second against another.

Most people don't have big data, but some people do.
I agree on the "default position", though I cannot see anything so inappropriate to justify a HR call or a complain to the (official or unofficial) company list, at the very most a comment like that could have provoked some snappy response and an immediate - on the spot - clarification between the parties, at least between "normal" (adult) people.
That would be a "normal" reaction as I see it, saying nothing then writing to the list a complain seems to me like the act of someone that either has experienced much more serious "name calling" (or some other untold offence) or that is a tad bit oversensitive.

Most probably normality - like beauty - is in the eye of the beholder.
Most patents use "operable to" or "configured to" to get around mixed apparatus/method claim prohibition.
I think if they would track an event that's only 2Hz that's too much time to unlock. People would complain that it's slow.
So long, Apple
Google begins selling Jamboard, its Surface Hub competitor
Roosters can be mean as hell. I used to help my cousins with their chickens and we would always have to lure and trap the rooster before we could go near the hens.
Maybe. I suppose it depends on the goal.

From my own experience, generally the barrier for being productive at a job was soft skills, in that if a team member was suffering from an insufficient hard skill but had good people skills/personality traits I could teach them enough of the hard stuff they are missing so they can be productive, but the inverse has rarely proved true.

Now, you can work around a team member's deficiencies in soft skills, but the difference is they aren't likely to pick them up.
Agree, though this guy is far from ethic hunting:
http://www.dw.com/en/widespread-tree-felling-sparks-outrage-...
Show HN: 15-question programming quiz with answers
This is really good proof of concept project. Blockchain is super exciting and I share your enthusiasm. Problems it solves are real.

I kind of feel it will not take off, no windows support etc, but you never know, if anything is hot these days, it is blockchain solutions.
No kidding. What about US customers that want "to manage all telemetry and updates"?
I dunno, JIRA works reasonably well as a standalone and integrates with everything under the sun. However I definitely see your point in replacing JIRA w/ Asana as it's really gutting the core of the suite.
Any chance these podcasts are transcribed somewhere?
Or alternatively Poland could be kicked out of the EU and UN for failing to abide by the laws and conventions that they signed up to when they joined. I think that would be fairer.
That's because there were no large slave holding stakeholders in England when they outlawed it.

Britain certainly had no problem importing and millions of pounds of cotton and sugar picked by slaves, even after the ban.
For decidability to apply, the input needs to be known a priori in its entirety. Decidability is a mathematical statement about natural numbers (or any other countably infinite set); it has absolutely nothing to do with "paused" input or any notion of "interactivity".

The type of behavior you're concerned with does on the other hand fall squarely in the realm of temporal logics (e.g. TLA+), which do concern themselves with interactivity and indeterminate pauses. For example, the statement "so long as the user eventually inputs the string 'Ralph', the algorithm eventually terminates" is decidable for any finite state machine.

In other words, if you limit your interactivity logic to that expressible by a finite state machine with decidably halting transitions, congratulations, you are writing decidably halting programs, even though they don't "halt" in the temporal sense. It's always possible to tell whether they will halt for a given input by completely analyzing the state space.
Like the OP said, check out Pandas which is ideal for this. Take a look at Spark. Also if you're using parquet get familiar with Apache Arrow.
We've been using a quiz as part of our hiring process (at Triplebyte) for the last two years. However, a lot of people do our quiz for fun. So we decided to create a version optimized for this (and hopefully a little education). The problems come from the most common mistakes / misunderstandings we see when giving interviews, and ramp up in difficulty. We put a lot of effort into writing interesting answers.

Let me know what you think!
The problem is, AlphaGo can read the "chances for winning" vastly better than any human being. AlphaGo may calculate a "chance to win" of 99.6% to 99.5%, and it will then select the 99.6%, even if that's taking 8 over 10. A human simply can't do that. That "close" of a winning probability looks the same, even to a master at whatever game they are playing.

This is literally why chess computers have gotten to the point that no one can beat them.
I am going to start learning Python, then eventually Django in a few weeks on a Surface. What is your overall setup for making it as painless as possible on Windows? I'd say my number 1 frustration in PC devops and I'm trying to figure out the install with the least amount of headaches.
Ask HN: First side project, What should he use?
Aside from google maps, are there any other great examples of mapping software that runs right inside a mobile browser?

I am helping a friend get started in a side project, and he is looking for the simplest possible way to start making something with maps that will work on the phone browser.
Edgar F. Codd just started spinning
It's likely Verizon using Javascript-based content injection to append that link to the footer. ISPs do this sort of thing all the time.
The business problem is that the world has increasing apathy towards computers.

“When we were an agrarian nation, all cars were trucks, because that’s what you needed on the farm,” Jobs said at [the] D8 conference in 2010. “But as vehicles started to be used in the urban centers, cars got more popular. Innovations like automatic transmission and power steering and things that you didn’t care about in a truck as much started to become paramount in cars. … PCs are going to be like trucks. They’re still going to be around, they’re still going to have a lot of value, but they’re going to be used by one out of X people.”
There are some SQL/JSON combinations that allow you to keep the unstructured stuff in a blob and promote some of the attributes as keys, etc.
> Windows had been banned from Chinese government computers in 2014 amid concerns about security and US surveillance.

Really? Which ones? I see Windows running in the police station, DMV, ... "Government" is a quite broad term.

What do they use instead? Ubuntu Kylin or their internally developed Linux?
Belovezhskaya Pushcha is one of the oldest official natural reserves in Europe. I can't give the sources right now but it was already protected area 600 hundred years ago. It had a special status since the early times of Russian Empire. Even Nazi Germany during its invasion to USSR recognized the importance of the forest and didn't sent the army through it and kept it intact. Also it is a home to bisons which were extinct in North America and recently some animals were transfered from it for repopulation.
Do you know of an example of someone doing one of these heritability calculations, saving the model, then plugging in data from a new dataset into the same model? (ie the new data did not exist at the time the model was created)
you are literally participating in social media right now by reading and writing these comments
TruthCoin style oracles look like they might work. I think they are worth trying but wouldn't be surprised if they failed.
No one, including everyone on HN, self taught or otherwise, "deserves" a job. Jobs are earned. You need to demonstrate that you can provide value to someone else directly (freelance) or to someone else's value producing firm (employment). The exchange of value (goods and services for goods, services or a tool that can be exchanged for either, money) is the foundation for why jobs exist in the first place. Without the exchange of value, the only job that exists is providing for yourself.
I find it hard to believe that there is no normalization that can be done in a thousand-column record. If you use any decent ORM, you can look at these records and find out what are the real classes/entities and their relations.

But if you really have some crazy single record with thousands of attributes, look at CouchDB. If you know your way around Javascript and don't mind writing map/reduce functions, it should be the easiest way for you to do reporting/analytics.

Or if you just want to avoid a database altogether and know some Python, take a look at Pandas.
Lincoln Said What? Bogus Quotations Take On A New Life On Social Media
> if you're hand wringing about whether it's fun to write then you're optimizing for the wrong things

It's hard to imagine what kind of career that attitude would lead to. I very much worry about whether things are fun to write. And I've been lucky that, so far, I've had jobs that were very fun. Not necessarily easy, but fun.
Great point. Ultimately, you need to create a situation where they will also be winning career-wise.

I recently wrote about this. The TLDR is that 20% time is a great investment and can ultimately save the company a ton of time and money. It gives the engineers some playtime in order to build their CV's and "get their wiggles out". Ultimately, if done right it can protect your production systems from a lot of madness. https://hvops.com/articles/how-to-keep-passion-from-hugging-...
If they like you, no whiteboard coding will be given. If they don't, you can do perfectly on whiteboard coding and still get rejected. This applies to all companies not just Amazon.
Same case. Ingesting 100s or 1000s of IoT "events" per second,pipelining them and reacting. Solved with redis pub/sub and python subscribers.HAd machines responding to system interactions in no time and could prototype a new pipeline in 5 minutes.

The big-data/"cognitive computing" unit of the company would have suggested using Hadoop, Kafka, IBM Watson , AI , neural networks and lots of billable man-hours to match my solution.
They even looked interested in making and aquisition for a company with an event processing engine that did basically the same: pipe stuff trough processes.

I recommend reading Event Processing in Action and then just building the same with your favourite pub/sub toolchain.
What makes RabbitMQ/Celery/etc. any "simple"r than Kafka? I found the experience very similar, and in a JVM-oriented environment Kafka was easier to deploy.
This page is https.
Why the notion of using machine learning to detect malware is completely daft
Common sense says that non-GC languages should beat GC languages, but there are actually cases where GCs can be faster. In particular, allocating memory can be just incrementing a pointer with a GC, whereas malloc does some more bookkeeping. You pay the price later during garbage collection, but if you are lucky your process is idle at that time, or even better, it is finished, and you get cleanup for free. See also https://stackoverflow.com/a/6977194/143091 What's faster will vary a lot from case to case.

(Of course, deep down the stack is always a non-GCed language, or at least assembly, so if your GC language wins a benchmark, you can always write that same program + the GC runtime in a non-GC language... but that is a trivial observation. Nim compiles to C, so is it GC'ed or not?)
PhishBreach – Was your organization phished?
If one has tons of workloads (like in hundreds to thousands) to run concurrently on a lot of the data, Hadoop is pretty solid.

YARN is also allowing interesting stuff.
Data are plural, eg: "Data arent't that big." I believe datum is singular.
A Man Who Made the Mistake of Trying to Help Wikileaks
To be honest, "this kind of work" was a euphemism for "Just what the hell are you doing that requires over a thousand columns?"

I understand though that sometimes customers give you a rotting dumpster of data and ask for critical insight into their operations.
A brief history of “type” (2015)
You have to say which set of programs you will run, and then you have to pick how you will describe a set of programs (since the name of a set of programs is just data).

Wikipedia lets malicious editors enter lies...these lies can be very difficult to detect so you need smart people to see them. The hope is that because it is text, people know how to read and can use critical thinking...computers on the other hand do not do this with code; they apply a set of rules that governs what can and cannot be executed; in other words, the input is either in the set of programs it will run (or not).

Now, you could allow each article to name a way of translating between...and all of a sudden we are kicking a hornet's nest of code-breakers :D :D :D

So, the short answer is...YOU COULD DO IT...but why would you want to attract the attention of people whose JOB it is to do figure out breakthroughs in code-breaking?

This gets into ZFC set theory and Tower of Babel and intelligence collection stuff. Essentially the problem is that if we want to define the boundary of what we will execute, we wind up getting marginally closer to state secrets.

And then, before you know it, somebody posted the code to crack the data-link to a drone in (redacted foreign country) and this caused the deaths of ___ soldiers.

So, you have to spend some time defining the territory before you let people wander aimlessly into danger.
Eye color in humans pretty closely follows the Dominant/recessive allele characteristics from classic Mendelian genetics. If you show me a person's family tree and I notice that all 4 of their grandparents have brown eyes I can predict with almost total certainty (barring new mutations or epigenetic effects) that they will have brown eyes.

other combinations of eye color at the grandparents lead to predictable distributions of the expression of that trait in the grandchild generation. this may be what was meant by "random variable", but it's really a population effect so modeling it as a random variable is peculiar. it's not random at the level of the individual, or at least it isn't after the recombination phase of fertilization in the embryo.

given a genetic sample of an individual it is totally predictable what their eye color will be based on examining their genes. I don't see how this is begging the question.
I can understand the product from the docs, but not necessarily why Kafka-everything seems to be approaching Hadoop-everything.
I always start with the test suite.

If I'm unfamiliar with the code base I start by trying to break things. When I do this I'm looking for missing assertions, invariants, or incorrect assertions. I create tests for those. I fix them. If the project I'm working on is smart enough to use property-based testing I look for weak invariants, serializability, or missing assertions. The end goal is to understand the module by exercising it and adding value at the same time.

If the code base isn't using property-based testing or has no tests at all I will find a way to add them. It gets the ball rolling for adding some issues to the tracker and provides real value.

Keep a journal of your experiences coming up to speed. Once you've grasped a certain corner of the code base review your notes and build some documentation for that code. It will help on-board new developers the next time they come to that code base.

As for introducing best practices, as a senior developer, you'll have to build consensus with the rest of the team. In my team the difference between a senior developer and a team lead is the ability to communicate ideas and build consensus around them. Learning how to influence people is the hardest part about building your skill as a senior developer. Senior developers become leaders when they can communicate a proposal and immediately build in feedback from the team and turn that into a real change to the code base, product, and business.
The other point that's commonly missed is that Hadoop is really only useful when both inputs and outputs are too large to fit on one machine. If you have a big-data input but a small-data output (which is very common in a lot of exploratory problems), you can get away with a simple work-queue setup that sends results into a shared RDBMS or filesystem.

At the beginning of my current project, I had a job that involved 35T of input, but the vast majority of records would be ignored, and then for each successful one, only a few hundred bytes of output would be generated. Rather than Hadoop, I setup a simple system where a number of worker processes would query Postgres for the next available shard, mark it as in-progress, and then stream it from S3 and process it. When they finished, they'd write a CSV file back to S3. The reduce phase was just 'cat'.

The resulting system took a few hours to build (few days, including the actual algorithms run), and it was much more debuggable than Hadoop would be. You could inspect exactly where the job was, what shards had errored out, and which were currently running on machines, and download & view intermediate results before the whole computation finished. You could run the workers locally on a MBP if you needed to debug a shard, with no setup needed.

When I was at Google, we had a saying that "The only interesting part of MapReduce is the phase that's not in the name: the Shuffle". [That's the phase where the outputs of the Map are sorted, written to the filesystem and eventually network, and delivered to the appropriate Reduce shard.] If you don't need a shuffle phase - either because you have no reducer, your reduce input is small enough to fit on one machine, or your reduce input comes infrequently enough that a single microservice can keep up with all the map tasks - then you don't need a MapReduce-like framework.
Maybe I'm weird but I find programming syntax much easier to reason about than SQL syntax.
One bunch of yahoos I ran into was building a Hadoop cluster to do some bullshit to user and PC configuration data before loading it into a big helpdesk system like Remedy or ServiceNow.

They were 6 months in and made no progress other than building 30 servers. I wrote an awk script to process the CSVs and did the rest in Excel in about 30 minutes. I had an intern automate the process with a perl script, which took 3-4 days! :)

The program management was very upset, mostly because they looked like a pack of clowns.
It's funny, truck drivers seem an ideal fit for unionization, but tend to skew politically anti-union (in the South and Midwest at least).

Agreed that drivers have responsibility for a load they pick up, but my understanding is that with the new eLog systems there's less opportunity for "flex time". So if you arrive at a yard and need to get a container swapped to a new trailer or have them pull a different trailer then you're still on the clock. Which gets dicey when the yard knows this and says "Sure, we'll get you a new trailer. It'll just take a couple hours. We're really busy right now..."

The ideal would be the unionized situation you outlined, where there's a strong, independent driver's advocate to mediate disputes.

And agreed that there's a hard cliff where no one should be allowed to drive. But if it's 11 hours in the seat max, then what if you sat in traffic at a port for 5 of those? Is that driver as tired as someone doing 11 hours of interstate miles?

Paper logs afforded both drivers and regulators a judge-like discretion that I think lubricated the wheels with reasonableness while still dissuading repeated patterns of risky behavior.

eLog (assuming they fix all the loopholes in the terribly implemented first gen systems) doesn't provide that flexibility.

Which, in general, is a problem I think we're going to have to deal with more and more. You can't take partially enforced laws that worked, implement technical measures to absolutely enforce them, and expect everything to be the same as it was before. (E.g. if police departments started sueing for Google Maps tracking data and handing out tickets whenever a speed limit was exceeded)
Half-Baked Solutions to Common RxSwift Problems
Gezonde voedingsmiddelen die je regelmatig zou moeten eten
I've come to know Kafka as very reliable and well understandable queue that solves a few things much nicer than rabbitmq - for example the ability to replay all events still retained in the cluster. My primary pain with it is running the zk cluster, but other than that it's my to-go queue for moderate or larger throughput queues where loosely ordered at least once delivery is fine. My reason to pick it is the feature set and that it has yet to let me down and munch my data.
Everyone I know just increments a number at the end of their password, until it allows them to start from 1 again.

ShittyPassword1
ShittyPassword2
ShittyPassword3
...
ShittyPassword8
ShittyPassword1

NIST came out recently against this, so hopefully, hopefully companies will start to listen.
How to Draw Mushrooms on an Oscilloscope with Sound [video]
I am thoroughly baffled why would anyone want to do this. I know it's all the rage but I believe it's an overhyped fundamentally bad idea: the priorities of the back end and the front end are so different that you need to think differently anyways.
You're right - it doesn't apply to culture.

The "corporate person" is being a sociopath as usual and rather than saying "it harms our company" they attempt to spin it into something that hurts the individuals that work at the company. It doesn't. It hurts the "corporate person", but they can't come out and say that.
Windows is probably the best OS on this form of PC. The form itself is shit, that's another matter.