skysight-inc/HackerNews huggingface dataset:
> … there should just be one full time engineer on it …

Agree with everything you just said, except, please make it two engineers. Speaking as a prototypical eng manager who typically inherits these types of projects when "that one engineer" leaves the company, I strongly believe that any job worth doing is worth doing by more than one person. (Plus, a good peer, just to bounce ideas off of and do informed code reviews with, is worth their weight in stars.)

PS: I thought @jack was going to open the third-party ecosystem back up again. That also has the potential to help address this problem. Did that not happen?

[1] http://www.theverge.com/2015/10/21/9586084/jack-dorsey-twitt...
I want to speak to one aspect of the wonderful article:

Listening to weak signals.

I'm about to do a shameless promotion for a book I have nothing to do with, but a book that has been a guiding light for me: Michael Lopp's Managing Humans

When I was reading the article I couldn't help but think of Lopp's advice about regular one-on-one meetings with each of the people on your team.

I think this is one of the points that Lopp intends for managers to be listening to during those meetings.

They aren't so much for feedback from the manager (as they are often treated), but more as opportunities for the manager to listen.

If I understand the book correctly, those one-on-one meetings are exactly the place where the managers are supposed to be listening for the "weak signals."

I am not an expert in every area of development, and yet I have somehow been inserted into a management role.

As Lopp explains very clearly, this happens often, and the single biggest thing you can do when that happens is care about being a manager. It's a different skill set than being an IC.

Recognize that, but don't get totally caught up in that. I don't think Lopp would disagree with anything in this article. I think, in fact, that following Lopp's ideas would lead to far fewer cases of WTF than what we see in the wild.
The article mentions that they heavily relied on open source software. Ever developer that contributes to open source enabled their success directly or indirectly. It's worth pondering.
up voted, those are words to live by. it's not victim blaming, it's the reality of that situation and having the presence of mind to just shut up and take it. Your turn to punch back comes once you are free or in court.
It's insane the ebook costs $89.
> So how do you induce the incentives to create such open systems?

Unfortunately, I have no idea. Obviously I can say "only buy systems that are open", but it'll take a lot of people to make a significant impact. Hopefully, when things die, we can say "I never had that problem with $vendor because their systems are open!"
Something to keep in mind is that it's becoming a trend that companies sell the systems at a loss (Notably, I think the only company that doesn't is Nintendo). Especially for the beginning of it's life, this is only practical if it's a closed system - Because if it wasn't, then they're essentially just offering the hardware at a huge discount to people who have no intention of buying any games, and the games is where they're going to make their revenue.

I don't know if Sony every actually said anything official, but I remember that being part of the reason why the 'Other OS' feature was removed from the PS3 - Places/People were buying them up and running Linux on them because they were such a good deal for the hardware.

I wouldn't want to bet money on my predictions, but I would say that if someone does build a platform like the PS4 that's open and can run Steam, it would cost a lot more for the consumer then the PS4 since it would probably be priced closer to the actual value of the hardware - And that would put off a lot of potential customers.

That said, there's an obvious chicken-and-egg problem in getting people to buy the system, and getting developers to develop for it, but having a cheaper system is definitely an advantage (Obviously, already being well-known to both developers and consumers is probably even more important). Regardless, if they had set the price of the PS4 at $1,000, it probably wouldn't be doing so hot right now.
By parser I mean any small thing that might parse text.

Since I tend to deal with distributed systems, I generally think in terms of blackbox tests as dealing with application APIs rather than function APIs, and integration tests as multiple system components being tested together.
I don't read it that way. Maybe it's because I've written some agendaless tirades like that myself, but to me, it sounds like he's just pounding out exactly what he's thinking on the subject. Can you point out the parts of the post that you believe are "indirect" or "adulterated", and why you don't believe they're sincere expressions of belief in context?
You're arguing in favor of unit tests.

> If you don't have a test for a given item of functionality; how do you know it works?

This would be an integration test

> If you don't have a test for your subsystem that covers the more common use cases; how will you tell if it broke when you updated one of the functions/methods it uses?

That would be a system test

> If you don't have a test for individual flows at the level of the entire system how will you know when you did something that broke the build outright?

I may misunderstand the question, but ... you compile it ?

> Testing is an engineering practice that lets you make changes to complex systems with confidence that the system is still doing it's job after the changes you make.

And if you have unit tests ... doing those changes is twice as hard as when you don't have them. Also if you test the "job" of the complex system, that would definitely not be a unit test.

This is what unit tests look like in large companies. What you actually find in the field

def add1(a):
return a + 1

def TestAdd1(self):
self.assertEquals(add1(2), 3)
self.assertEquals(add1(-2), -1)
self.assertEquals(add1(8), 9)
self.assertEquals(add1(12), 13)
self.assertEquals(add1(5557), 5558)

Think

None of your concerns apply to code like this. And nobody, ever, for any reason, should write code like this.
Awesome article! This is the exact kind of use case we've been helping enterprises with at Graphistry, especially for SIEMs and operations data. Worth adding two aspects we've been finding important in our journey here:

* We found the need to play nice with Neo4j as well other more common systems here like Kafka/HDFS/Spark, Titan, and Splunk

* It helps to be able to work with big event graphs, where we'll often want to do something like filter for the day's 1M+ priority 10 alerts and see how they connect. The result is we spend a lot of time on our GPU frontend+backend so you can spot patterns in all of the days big events, and exploratory tooling so you can drill down rather than write queries.

If relevant, happy to share an API key (info@graphistry.com) or get on Skype!
I think I understand what you're saying, but I think you're drawing the wrong conclusion here. You just stated that services break by default. I think that's what the author is stating, too. Services break. People make mistakes. Failures happen. By accepting screw-ups we arm ourselves with a mindset that's focused on moving forward, instead of beating ourselves up over what's already occurred.
You mean, links like archives of his now-removed tweets where he himself is imploring us to investigate what's happened to him?

Nonsense. Absolute nonsense.
Pharma bro Martin Shkreli's drug company files for bankruptcy
GitHub Year in Review
Well if you're satisfied with the outcome then I guess it's good.
The references here seem to disagree:

https://en.wikipedia.org/wiki/The_Jewish_Race

Particularly, this statement: "Genetic studies on Jews show that most Jews worldwide bear a common genetic heritage which originates in the Middle East, and that they bear their strongest resemblance to the peoples of the Fertile Crescent.[58][59][60] The genetic composition of different Jewish groups shows that Jews share a common genetic pool dating back 4,000 years, as a marker of their common ancestral origin."

That Jew/Hebrew isn't a race is a very new claim to me. What do you have to support it?
He was doing good physics the whole time, even when his ideas weren't testable.
Yup -- they even have configurable SMS & email alerts you can set up to let you know when the water level at a particular point goes below or above a specified depth: http://water.usgs.gov/waternow/

I use it to let me know when my local river is running deep enough to make for a fun day out on a kayak.
I totally agree with that. I can throw a 100MB ZIP into my Dropbox folder without a second thought. Box and Google Drive are equally snappy.

With Google Fiber "Free" and Time Warner before it, I was capped at 1 Mbps upstream and cloud services were much more or a chore.
Just started to check it out. I'm looking forward to reading the rest of the book. I've found the common atheist arguments (ie.. people start wars in the name of religion, therefore religion is bad) usually pretty weak.

The first part looked interesting. I wanted to see exactly what he is attacking, and it seems to be specifically the idea of the supernatural. He notes that Einstein has been misrepesented as a believer (didn't know that - so I learned something already!).

However, I would expect a physicist, engaged in deep study of the physical world - to deny the existence of the supernatural. Doesn't accepting that kinda ruin the whole basis of their life's work? Dawkins mentioning Einstein didn't believe proves nothing. Einstein not believing something doesn't make it false.

Ultimately, what it comes down to, at least for me is - we know little about the origins of the universe, not to much about even our own minds, and nothing outside of our consciousness.
Good catch. Mental slip on my part during a hasty reply.
Entire US voter registration record leaks (191M)
I vote yes.
Someone needs to explain the whole Amiga thing to me, because I've lost the thread. When you say markets might number in the hundreds, I have a hard time believing even that. Perhaps I'm just ignorant (as I said, I've lost the thread), but why would anyone run Amiga OS any more other than for the novelty? Are there still applications that are worth spending money on?
Wow, this is very generous of you, thanks a lot! They all look applicable to me, as I was using this script to try out some Python with a view to doing more complicated things later.

I will pick up on these and let you know by email if I have any further queries.
The biggest problem is that the moment I touch the screen the page redirects to advertising and malware.
> Why do developers choose a closed platform like PS4 vs the Steam Machine?

Several reasons, in order of significance:

* Console manufacturers will often subsidize your development for platform exclusivity, in order to attract users.
* Large userbase.
* Single hardware target. While consoles (intentionally) have no software portability, you are also targeting a single set of hardware so the QA is easier and the dev process requires less testing.

So basically, in reverse order, if you want Steam Machines to win:

* You need APIs that are perfectly portable, so your SDL + Vulkan game can run everywhere without having to manually fix everything.
* You need users buying your platform.
* You need current manufacturers to lose their profit margins enough to stop buying off exclusivity deals.

Note that with consoles, exclusives attract users which attract exclusives. Its a positive feedback loop that, at least in this console generation when you can build a PC of comparable power to a PS4 / Xbone for the same price, there is no barrier to entry cost on the user besides the preassembly costs, and the fact Steam Machine vendors are selling at profit where Sony / MS care less about margins because they make per-sale revenue in the same way Steam does. But Valve isn't making the consoles, so Alienware needs to keep the lights on exclusively through sales.

That means there is both, A. no incentive for platform exclusivity on Steam Machines, because you aren't getting lockin to your Steam Machine, you would buy exclusivity to the ecosystem and B. no Steam Machine competitive with PS4 / Xbone in price / performance unless you are buying a lot of games to make up the gap between console used prices vs Steam sales.

> Is there a future with an open gaming platform like the Steam Machine?

Yes, but not without Valve biting the bullet to force enough user adoption to get the feedback loop going. They launched it last month without any compelling reason to buy one, and thus sales were underwhelming. And the only way to make them a compelling buy requires Valve to spend money on it - either through subsidizing sales with huge amounts of free games, or directly by making Steam Machines more cost effective up front than current consoles. And then on top of that, you need a critical mass of users to get developers releasing all cross platform releases for it, while also having the games to attract users. Usually, you get that through exclusives, but as I said, nobody has a motivation to release exclusives on SteamOS except Valve, and that would only be if they value adoption of their platform over raw sales figures, and I bet they don't.
Yep, it was just a sub-optimal word choice, a bit too strong.
The desert dollar industry (2021)
191M US Voter Registration Records Leaked in Mystery Database
I have a friend who went through much the same process. I converted him to full-time Linux usage, and he stuck with it for a good 5-6 years, but eventually went back to Windows as his host OS so that he could conveniently play games, connect projectors, modify photos and video, and do all of the other basic things that only Windows and OS X users get to do reliably. He still does all of his development and sysadmin work from a Linux VM that runs on his Windows host.

I've considered doing this myself, especially with WINE taking eons to implement adequate DX11 support (most games released in the last 3 years won't work), but it's a plunge I haven't been able to make yet. I think I'm going to just set a Windows box right next to my Linux workstation and move the mouse, kb, and monitors over when I want to do something Windowsy.

The ideal would be getting hardware that supported raw GPU passthrough, getting a spare GPU, and doing all of the Windows stuff in a VM (this allows you to get 90-95% of your full GPU performance, and have the card controlled by the Windows system), which would really be the ideal solution IMO. However, I found that even the full technical manuals rarely mentioned if the hardware had the IOMMU, the require capability (both the processor and the motherboard must support), and that by far the vast majority of hardware doesn't.

I have a dual-boot partition now, but I use it probably about once every 9 months, and mostly just to poke around for a second and see if it has spontaneously combusted whilst unsupervised before rebooting back into my workstation OS. It's simply too much hassle to have my primary workstation offline for hours at a time while I game, because it hosts many services that my house relies on. I also have a Windows VM that I do most of my photo editing in.
In python 30K is not as small as say 30K in Java
I better call my local representative, I'm going to be rich any day now...
As a concrete example: we work with enterprises with people numbering anywhere from 10K to 500K to government-scale, and each person may have a desktop/laptop/phone, and all the servers/printers/switches those connect to, and at the logical layer, all the applications and services for making it useful. We'll see multiple central logging systems, hierarchies of administrators, and the results of mergers, acquisitions, and one-off or zombie projects. These organizations are getting sophisticated enough to log 10M, 1B, etc. alerts a day (ex: using graylog or splunk), so we need to focus on the next step of being able to point to one alert and asking what's happening around it.

It's a really fascinating data problem, so we've been loving building tools for seeing into it!
> the Office has conducted “thousands” of field tests and the only false positive results of which he is aware are the results at issue in this case.

I'd imagine most cops know what marijuana smells like and use the field test to back up their correct observations. I imagine they do very little testing of other random things to see what causes a false positive. We need better statistical education for the legal profession.
I agree!

Donella Meadows is one of the most articulate writers and thinkers on systems. She is the easiest to learn the basics from: http://www.amazon.com/gp/product/1603580557

Same person, shorter, free, and condensed format: http://www.donellameadows.org/systems-thinking-resources/

Same author again, the final chapter from the book above: http://www.donellameadows.org/archives/dancing-with-systems/

And if you only get one book on how to apply it to business, it's this one: http://www.amazon.com/Leaders-Handbook-Making-Things-Getting...

A more recent applied-systems-thinking manual for organizations that I've found hits home with more traditional managers (very useful): http://www.amazon.com/The-High-Velocity-Edge-Operational-Com...

Another good one that's a bit long-winded, but another set of applied examples: http://www.amazon.com/The-Fifth-Discipline-Practice-Organiza...

Senge, in the foreward of that book, gives almost all credit to W. Edwards Deming—who is the originator of many of the ideas of organizational systems thinking and how it integrates with Management. So, if you want to go deeper, Deming's book "Out of the Crisis" is a good tome.
The worst part for me is this he said in kickstarter:
"These moves also allow us to ensure Console OS will always be free, via GitHub. When we sell Console OS, we plan to commit most revenue right back into open-source development. And sales of Console OS will not be in place of our GitHub distribution - it will be right alongside them."
So he might sell a fork? Wow, well that's silly.
But we had music piracy before 1mbps , only limited. And we had video torrents at before 10mbps(and at low quality), so those use cases we're clear, and we could have guesses that maybe legal versions of those activities will also appear.

But currently the limits are more tied to physiological limits - the highest bandwidth channel for a human is the eyes and we know their limits (somewhere between 1080p to 4K depending on various things), and we've almost covered that.

The only open question is VR streaming, but i wonder whether this work considering the extreme low latency required ?
What does this translate to in terms of dollars from the 400 so called high earners? And then what portion of US overall tax revenue is from that group vs other segments? Any such data broken out like that exist?
One should always keep in mind that a WTF-level issue is very possibly a critical oversight on the issue-holder's side. Lots of things in the world appear very fucked up, but are in actuality slight improvements over even more fucked up states of affairs. One should endeavor to find out the history of the situation before jumping to judgment.
That looks pretty neat. Can you route and firewall at gigabit rates?

Also, it looks like it would cost ~150->300 USD to make a complete (board, case, storage, AC adaptor) router?
Red states are wrong about many issues but gun control isn't one of them. Cops think twice when they know there's an equalizer in the equation. If you live in a part of the country where the populace is disarmed, don't be surprised that police are beating the shit of you, or terrorists are targeting your gun-free zones. Exercise your 2nd amendment rights, and open carry that AR15 with 30 round magazines proudly.
>Microsoft and Apple both have many employees and lots of money invested compared to Linux

There are a lot of people who are paid to work on the kernel full time from Red Hat, Google, IBM and many others. If I had to guess I'd say there are probably more than the other two, it would be interesting to find out. But if you include people where it's not 100% of their job, but still an official part of their job, I'd say it's almost certainly more for Linux (not even counting unpaid contributions).

Linux is the most popular platform for servers and HPC because most of the time it's the better kernel. It's so dominant that Apple has basically left the server area and the handful of showcase supercomputers built on Apple gear have long since faded from view. Linux went from having one supercomputer in the top 500, a fraction of a percent, in 1998 to 98.8% of the top 500 currently.[0] The other six seem to be IBM machines running something else. OS X's first release was 2001 and it and Microsoft offerings are simply not present in that top section of the HPC space.

Linux is also the kernel on the most popular smartphone platform so it's not all computational either, when the hardware is tightly controlled it works fine. The problem isn't the number of developers on kernel or how hard it is to debug, it's that laptops and desktops aren't offered from a single company that can tie everything together.

[0]http://www.top500.org/statistics/details/osfam/1
Considering it has been about two decades since the Amiga platform endured Commodore's collapse, that it exists in a way that people care enough to fight over it is pretty amazing.
Are there any actual investors in Los Angeles?
I live in LA and I'm not moving. I've been a web developer for 15 years. I've risen to the top of my field and worked at the largest corporations in Los Angeles. I have what I think is an incredible and novel idea. Is there anyone I can talk to in Los Angeles? I love my city and I'm close to my aging mother. I'm not leaving, but there is a lot of money to be made with this. Will we be forever ignored in Los Angeles? Are we on our own?
They've already tricked everyone into dropping the better plan with free upgrades ever 2 years in exchange for a hike in data service fees... best marketing snowjob ever. Plus, with their "pay the phone off a month at a time" crap almost everyone opts for they still effectively have contracts.
> PS: I thought @jack was going to open the third-party ecosystem back up again. That also has the potential to help address this problem. Did that not happen?

I asked Jack about the third-party ecosystem on his producthunt Q&A, he said Fabric is a strong offering and they're building more to repair relationship.[1] I don't know if them building developer tools is same as repairing relationships though.

[1] https://www.producthunt.com/live/jack-dorsey#comment-202231
Google provided both modems and routers, which was absolutely essential. Very few people had/have the hardware to take advantage of the fiber speed.

Anyone who wants to roll out a fast service should certainly take that cue.
Please donate to FreeBSD, they are only 62% funded for 2016
Happy GNU Year Last chance to give in 2015