skysight-inc/HackerNews huggingface dataset:
I think that author just does not have experience outside of what he does. Maybe his systems can fall back to sane defaults. But what is sane default if you have to communicate with 3rd party server and your system is worthless when connection is not there? You have to have ip/domain name configured.

For cars if something is wrong then in some cases you can start and even drive but users get warning. If there is something really wrong car will not start.

So I think what author suggests is at least asking for trouble.

Almost everywhere as quoted:

"Everything in this post is about improving the deployment process for your applications, specifically those running in a Docker container, but these ideas should apply almost anywhere."
It helps but doesn't quite answer my question yet.

With your situation #1, what if this is very common transaction and therefore you have 100 of these all waiting. What about 1000, 5000 etc. what system resources are used to let these transactions wait indefinitely ( if I understand your semantics with specific regard to blocking )?

Some systems handle this as a failure that is communicated to the client rapidly. Other systems let N clients actually wait indefinitely but at the cost of taking up a thread / file descriptor, etc. in systems that have finite amounts of threads for example this would then be communicated in his paradigm as a such of upper bounds as to how many requests one could have waiting.

So just trying to get a feeling for how this could have infinite amount of waiting transactions due to partial failure and still keep taking requests.

Thanks for the reply, this stuff is always interesting
There is a lot of stuff the human eye cant see that is very interesting. One of the challenges in medical imaging is getting accurate labelling of images. For many labellings we see large inter- and intra-observer variability. We have both the problem that humans see something that is not interesting and miss something that is interesting.

I currently work on estimating emphysema extent in CT lung scans. Emphysema can be very diffuse and it is not possible to label individual pixels, so instead we try to learn the local emphysema pattern from a global label. Neural networks are interesting for this problem because the learn the features, but it is also a "problem" because the features might not make physically sense, which could make it hard to transfer the model and convince clinicians that they should use it.
I don't know anything about Docker, but to use iPython Notebook, you need to open it in a browser. The command line starts a local web server. It says it is running at "http://0.0.0.0:8888/" in your case (which sounds invalid). Usually, Notebook opens the browser for you automatically when it starts. I guess Docker interferes with that.
"No one has been able to suggest any possible bad consequences of wiping out all human disease carrying mosquitos"

This was not the question.

But I see that "unlimited" risk is no problem for you. The problem I have is that I am sharing this risk for your actions. How about to try to get this insured? I am curious about the price quote for an "unlimited" risk that an insurance company would quote.
The Surprising Failures of 12 Steps
>> I want the language to be verbose and easy to understand.

> So, you basically want to write in assembler, which printed is very verbose, and it's very easy to understand each separate instruction.

Please don't do that: You took what he said, jumped to the most extreme case, and then attributed it back to him. He never implied that he wanted to write in assembler. In fact, based on the part you didn't quote, I think he would not want to work in assembler.
Airblade/acts_as_enterprisey
It really depends on the kind of parsing that you're doing.

Full context-free grammar are supported by "generalised parsing". The older stuff is GLR by Tomita, Early parsing, the CYK algorithm. The newer stuff based on Tomita is the particular rabbit hole I stuck with for a while. I read about SGLR which eliminates lexers, GLL which is GLR but based on the LL algorithm. The people who do GLL research also did improvements on SGLR with improved speed on right-nulled grammars. Then there is the SGLR improvements with disambiguation filters, automatic derivation of error recovery with island grammars, etc. The disambiguation filters include a kind of negation, making the current implementation of SGLR for SDF capable to parsing Boolean Grammars, which are a superset of context-free grammars.

Anyway, there's more than context-free grammars. Definitely look into data-dependant grammars! It's able to define a lot of interesting things, like network protocol formats where you parse the length of the payload, then based on that length you know how much to interpret as the payload before you read the footer of the message. And you write all of that in a more declarative way and get a nice parser generated from it.

There is so much more, but I think I should to stop now :)
Bumblebee Flight Does Not Violate the Laws of Physics
Zeroing in on one book is crazy. However singling out a single author each week might work.

One week it's Hemingway and the next week perhaps it's Elmore Leonard. You decorate the store and the key is the author has many books and you stock them all. In a place like Tokyo, NYC or London this approach might work.

It would almost be like an art gallery highlighting dozens of paintings by a single artist. You could invite people in to discuss the author and those events would draw in even more people.
High Performance SSH/SCP – HPN-SSH
ctrl-double-click or ctrl-right-click > open
I'm more-or-less doing this now too, here's a few problems I've run into:

- Visa renewals. Leaving the country every 90 days to renew your tourist visa gets old fast.

- Finding a country that is both stable and has a low cost of living. I was in Bangkok during the last revolution when the current military junta took over. The airport shut down and I had to take a bus past a lot of guys with machine guns to get out of there.

- Starting a family changes things. Poor countries have poor education systems. I enjoyed my visit to Pattaya, but wouldn't want my kids passing by "walking street" on their way to school.

- You lose some small niceties like Amazon Prime and Hulu :)
tqdm is failing for me on Windows at the moment. To be fair, it might not be its fault (I'm mixing it with Blinker signals), but still I'm slightly disappointed.

A lot of these "magic" tools fall apart when you're trying to do something slightly more structured than "throwaway bunch o' functions".
A Distributed Data Processing Pipeline with AWS SQS and Spotify Luigi. Part II
dup : https://news.ycombinator.com/item?id=10782897 ( now: 244 points by antouank 21 hours ago )
He mentioned Sieve in passing, I wonder what the problem with it is. I use Dovecot and Sieve, and have never had a single problem, apart from mis-written sieve rules. The syntax isn't the most concise, but how much worse could it be compared to a hand-rolled sorting program?
Sorry, but it's not only Docker using 'containers' that way. I'm no fan of systemd for various other reasons - but that is one thing it does correctly: use namespaces aka 'containers' to separate processes.

It simply makes no sense to add additional unnecessary overhead and complexity to something that is essentially very lightweight. If you want a full-blown OS - a VM is much better suited at that, and modern hypervisors come with a ton of bells and whistles to help you manage full-os environments.
You may dislike this form, but it shows exactly why stevoo's opinion is formed
on wrong basis.

> In fact, based on the part you didn't quote, I think he would not want to work in assembler.

I have hard time to understand what he meant in this sentence:

> Dealing with thousands of line trying to figure out what is going on there it is essential to easily pick up what is going on !

Is it a praise of having plenty of code for a simple thing? Or is it a rant?
What makes you think there is funding involved?
The Rise and Fall of ‘Boring Dystopia,’ an Anti-Facebook Facebook Group
C doesn't turn into some magical obtuse assembly with optimizations enabled, a loop will still be visible in some way but might vectorized, unrolled, eliminated or even merged but you can still map it to the source. It's interesting to see what a compiler can do: ICC does some really fancy stuff such as eliminating whole chunks of code that are meant to help most compilers vectorize.
"Mac App Store customers running OS X Snow Leopard (v10.6.8) will be unable to purchase new apps or run previously purchased apps that utilize receipt validation until they install the OS X Snow Leopard update which will be available via OS X Software Update this January."

Note that Mac OS 10.6.8 is the most recent OS version for the Intel Core Duo Macs.
Those are the first MacBook (early 2006), the first 2 MacBook Pros (January 2006, May 2006), two Mac Minis (February 2006, September 2006), and two iMacs (January 2006, July 2006).

It also affects people who keep Snow Leopard 10.6.8 for sentimental reasons. Apparently 1 in 5 Macs use Snow Leopard. Not being able to receive new app updates could be troublesome.

http://www.computerworld.com/article/2487996/malware-vulnera...
To be fair this was 2007, they were interesting things at the time that you might plausibly muck around with for a side project.
> It is much more memory-efficient than thrashing a single graph over and over again. Rewriting and compacting trees is good for cache locality.

To be clear: if you are thrashing a single graph over-and-over again, you are doing many passes. Rewriting and compacting trees is just a way to make the multiple passes a bit less painful.
'Nothing'? Well, except a fuckload of experience, probably. Either spend 4 hard years at a top university or 10 years of consistent, hard work doing it all by your lonesome, maybe with and maybe without the benefit of being able to 'practice' at work. I don't think all it take is 'belief'. There is some actual hard work to be done.
This paper uses the open-source Rocket-chip processor (https://github.com/ucb-bar/rocket-chip), which is the same processor being used (and modified) by lowRISC. So hopefully you can buy RISC-V in the not-to-distant future!
I always notice how badly the phone mucked it up after the edit rule passes... That should be:

"The idea of doing tons of compiler passes works well in a functional programming context, where functional separation is the preferred method of abstraction, and you can use composition to effectively transform all your passes into a single pass."
There are tons of anecdotes i remember from AI classes about neural nets learning something other than what you expected them to learn from your training set.

For example, one story involved training a classifier to recognize an overhead image with tanks vs without. It turns out it ended up learning which days were sunny and which were overcast.

This sort of thing happens when training people from examples too: From the mundane cases in school, to the AA587 crash in Queens NYC.
Huh so I guess those macs are, to some extent, not end of life?
Here's what I believe: startups are for "fun" money. For "real" money, you'll do far better doing a traditional business where you're guaranteed a return on investment.

I'm talking about consulting, franchises, etc. You won't hit it out of the ballpark and TechCrunch won't feature you, but you sure as hell will have money in the bank.

How you use that money is up to you
I love the ideas behind Rust. I don't think it can replace C for all use-cases.

C has the use-case of zero-dynamic-allocations. AFAIK, Rust is not very compatible with that mode of use.
I do remember reading about qutebrowser; in fact, I did try it around three or four months ago. Might give it again for a couple of weeks and see how much I like it. Thank you for the tip.
It's both increased bandwidth and lower energy. It'd be useless if it blew the thermal budget to achieve the higher bandwidth.
I see no reason why it couldn't run in a separate container. You'd probably have to mount the postgres socket directory and the WAL archive dir into it, but it could be tricky - true. But containers are just a tool. Some things are not suitable to run in containers, don't try to shoe-horn everything into them.

Other than that, there's no problem running postgres itself in a container - as long as your data is stored in a volume ending up being bind-mounted on the local disk, and not on the layered filesystem - otherwise performance will suffer badly.

And yes - orchestration - especially on small-scale - is still a sore point. All the tools like kubernetes seem to focus on large scale and scaling multiple instances of the same containers - which is not what I and many people need. Something like docker-composer, but in daemon form would be nice.
Where did I mention vanity as a part of the raise? You missed all of the points I made and then attacked a strawman.

When you raise on your terms, as these guys probably did, you get (1) a great valuation (i.e. cheap money) and (2) great terms (i.e. 1x or better liq pref).
That's great!

Btw, if anyone hasn't seen the scene from the Simpsons i was referencing check out: https://www.youtube.com/watch?feature=player_detailpage&v=xs...

Simspons were ahead of their time apparently.
The more interesting take I've read on this is that the financial planners who instituted the Euro certainly saw this coming, and were using it in order to force through fiscal integration a decade or two down the line once they had achieved crisis conditions necessary for massive change.
this. so much this. after spending 2 years being forced by my local court to attend 12 step meetings I absolutely can attest to everything stated in this article. It is a religious ceremony disguised as addiction treatment.

I was raised Jewish, however I no longer identify with any religious affiliation. Being forced pretty much at gunpoint to attend Alcoholics Anonymous at churches where they end every single meeting with the people at your table standing, holding hands, and saying a christian prayer out loud with each other, was absolutely abhorrent. The one time I declined to participate in this hand holding I was getting looks full of daggers from everyone who noticed.

Until this recent experience with the judicial system, I still had faith in law enforcement. Now, I think we'd be better off policing ourselves.
Lara Joy Körner Nude Scene in Das Geheimnis Des Rosengartens
http://watchscenes.com/lara-joy-korner-nude-scene-in-das-geh...
The original source for this puzzle was the 2013 MIT Mystery Hunt:

http://www.mit.edu/~puzzle/2013/coinheist.com/rubik/a_regula...
At first this looked a bit weird. It may be a bit more land intensive, but the idea is sound, and would be able to expand to full capacity gradually. I like it a lot better than the hydraulic storage for the near term.
I can live in Germany for that much, and it includes health insurance. Are you sure that's the lowest you can go?
Javascript seems like a great contender in this space to me. Btw, as well as using nodejs / v8, there are also Rust bindings for SpiderMonkey (Mozilla's JS engine): https://github.com/servo/rust-mozjs

(not super user-friendly yet, as this mainly exists for Servo - but there's enough here to embed Javascript scripts in your Rust projects!)
Some more background:http://m.huffpost.com/us/entry/954642
I would just caution you that, node/JS has a much greater learning curve when starting out. However once you have gained knowledge of the different libraries (i'm thinking Promises, Async, lodash) and asynchronous patterns, you can write very powerful code in very few lines of code.

Python on the other hand; you will be able to get up and running, and implement large portions of the functionality that you require in your script, very quickly. The downside comes when you need to worry about optimisation. Without a lot of experience, it can be hard to find out how to go about making Python code fast.

At the end of the day, both are great languages, and have great communities.
> Here's a schema I use in production, see how readable it makes the parameters of the API and how quick all the validation and normalization is: https://www.pastery.net/mhwwnv/

Thank you so much for providing this example. I couldn't grok what schema did, and your code made it make sense.

If people want to provide examples for the other libraries in this thread, you'll be popular :)
Amazon plans to bring AR to any room
> With your situation #1, what if this is very common transaction and therefore you have 100 of these all waiting. What about 1000, 5000 etc. what system resources are used to let these transactions wait indefinitely ( if I understand your semantics with specific regard to blocking )?

So, as each client can only have 1 outstanding txn at a time, this requires there are 100, 1000, 5000 etc clients all submitting the same txn at the same time, and presumably they're all connected to the same server node, and for each of those txns, the set of server nodes that need to be contacted has been calculated, and the necessary messages sent. At that point failures occur so it's not known who has received which messages, so all these txns block.

The only system resources that are held at this point is RAM. Yes, it could get so bad that you eat up a lot of RAM - this is release 0.1 after all. There are a few areas where I have tickets open to make sure that goshawkdb can spill this sort of state to disk as necessary, though it may not be too horrible just relying on swap to start with.

> Some systems handle this as a failure that is communicated to the client rapidly. Other systems let N clients actually wait indefinitely but at the cost of taking up a thread / file descriptor, etc. in systems that have finite amounts of threads for example this would then be communicated in his paradigm as a such of upper bounds as to how many requests one could have waiting.

The problem is that in this particular case, the txn could have committed, it's just the server who initially received the txn from the client and then forwarded it to the necessary server nodes can't learn that it's committed due to failures. Now certainly I could make the client informed that the outcome is delayed, but the client may not be able to do anything with that information: it can't know for sure if the txn has been committed or aborted.

The entire codebase is written using actors and finite state machines. Because Go supports green threads and go-routines can be pre-emptively suspended, there is no problem with eating OS threads. In general, the design is such that the only thing that should block is the client and on the server the actor/state-machine that is talking to that client.
It's just a smart move to have the API your webmail uses be a "standard"...

I don't understand why i should implement Fastmails web API, when there are several major protocols that are supported by a huge ecosystem already: IMAP, CalDAV, CardDAV.
I agree. Things become incredibly murky if preconditions are not clearly seperated from optional settings with sensible defaults. For a server-side multi-user application to just go off hunting for data stores or even create new ones whenever configuration settings are missing seems like a security and data integrity nightmare.

If something is a precondition then the app shouldn't act like it wasn't just to make Docker configuration easier. It needs to fail fast.

Retrying database and other connections is sometimes the right thing to do in long running applications. But I dont' think application launch is the right time for it. Application launch is an opportunity to make sure that all dependencies were in place at some point. If things break later on, the odds of it being a temporary issue are much better.
Exactly what I tell my startup friends. I know people who run design consulting firms and pull in $300-500k/year in profits. Some run SEO/SEM marketing agencies and do similar figures. They give themselves generous salaries and own nearly 80-90% of their businesses which, on paper, are worth $4-5M.

That's definitely very, very well-off, if not F-U rich. And it's very "safe", unless your $1M in revenue hinges entirely on one or two clients
I think one of the major reasons for our disagreement is you perceive that there is an unlimited risk with GMO organisms - the paper you linked to explicitly states this [1]. Quoting from it:

Ecologically, in addition to intentional cultivation, GMOs have the propensity to spread uncontrollably, and thus their risks cannot be localized. The cross-breeding of wild-type plants with genetically modified ones prevents their disentangling, leading to irreversible system-wide effects with unknown downsides. The ecological implications of releasing modified organisms into the wild are not tested empirically before release.

The problem with this analysis is their assumption that GMOs have a propensity to spread uncontrollably. This is totally false. The problem with GMOs is they don’t spread very effectively unless under human selective pressure. The reason why is really simple - all organisms have a fixed metabolic budget which they can allocate to different functions (grow, reproduction, protection against predation, repair, etc). When you introduce new genes into an organism you have to take away some of this fixed budget to allocate to the new function. This means that when the GMO competes with other organisms in the wild they are at a disadvantage. If you make an argument based on a false assumption then you will get a faulty conclusion.

It is a real shame with this paper as the rest is quite good and an excellent argument for not doing things where ...the consequences can involve total irreversible ruin, such as the extinction of human beings or all life on the planet. GMOs just don’t happen to be one of these things (I am leaving out of GMOs things like creating new pathogens, but you don’t need genetic engineering to create these as the USA and Russia showed in the 1950 and 1960s).

The bottom line is GMOs are not that scary. In fact I know of no serious harm that has come from any other than to make people fearful for no reason and their absence to drive up the price of food.

1. http://arxiv.org/pdf/1410.5787.pdf
Enforcement Policy Statement on Deceptively Formatted Advertisements [pdf]
Merry Christmas, Happy New Year, Webix 30% Discount
Show HN: Algolia (YC W14) Presents DocSearch
>> No well, written software won't spawn zombie-processes - sorry.
> And yet it happens.

Strange, I have been running software in docker for almost 2 years in production on 6 docker hosts running a ton of containers these days, and yes - a lot of this software spawns child-processes.

In all this time I have never seen zombie processes with one major execption: Phusion Passenger to run our Redmine instance. If you run this under supervisord as 'init' process - you indeed notice the init process cleaning up "zombie processes" at startup like this:

2015-12-24 01:00:32,273 CRIT reaped unknown pid 600)
2015-12-24 01:00:34,774 CRIT reaped unknown pid 594)
2015-12-24 01:00:35,802 CRIT reaped unknown pid 610)

So that case for me is the exception, and I do use an init process (supervisord) to run only apache with passenger. Note that using Apache with PHP or plain does not leak zombie processes.
Also, the Guinea worm eradication program is getting quite close to succeeding. (http://www.healthmap.org/site/diseasedaily/article/guinea-wo...)
Zdrasti, Sofia CloudFlare’s 73rd Data Center Now Live
MTV's “The Challenge: Battle of the Bloodlines” (Cornessa) Summary and Review
Tinder Should Probably Fix This
This is where we break out the old Zawinski quote:

Some people, when confronted with a problem, think
“I know, I'll use regular expressions.”
Now they have two problems.

I actually had to tell someone this who wanted to build a machine classifier. They wanted the classifier as a step in getting data for their research. The problem was as far as I knew in the literature building the classifier would be a research project of its own.