skysight-inc/HackerNews huggingface dataset:
Several Rails apps I develop have been suffering from similar issues. Perhaps 2-3% of requests take 0.4-2s in just processing. If the allocation is a little intelligent, it'll not perform too badly and is less work than much harder optimization. Yet if it's random, it'll queue up horribly.

I'm pissed. Spent way too much time unable to explain it to coworkers, thinking I just didn't understand Heroku's platform and that it was my fault.

Turns out, I didn't understand it, because Heroku never thought to clearly mention something that's pretty important.

Easiest fix: moving to EC2 next week. I've wanted to ever since our issues became evident but it's hard to make a good argument from handwaving about 'problems'.
You should check the article in one of the comments in this thread - Israeli methods actually reduce security overhead. If there's anything the U.S is good at, it's logistics - I'm sure they can scale it just fine.
> Female devs are considered a novelty and the term female dev is considered to be frustrating due to the term dev being appended by 'female'.

The amount of active segregation is indeed frustrating.

> Why can't it simply be accepted that programming is simply a career that tends to appeal to men more than women in the same way that nursing appeals to women more than men?

Because programming was a field dominated by women until established as an academic field. Afterward, even then it saw a unprecedented rise in women in the 80s only to be followed by a sharp decline while every other math/science field had continued to rise: http://www.aip.org/statistics/trends/reports/spring07a.pdf You may fault the following as this is purely rhetorical albeit reasonable, but I find it difficult to believe that thousands of women who would have potentially studied the field suddenly decided upon a generational line that CS is an uninteresting pursuit. I align myself with Sapna Cheryan's conclusions of the stereotype threat in computer science as an explanation.

> I've also yet to come across a single, official complaint of sexism in the workplace within the tech industry.

I've come across many. The vast majority of aggressions I've witnessed go completely unnoticed and unreported among those that could do something about them.
> I feel that copyright infringement is stealing

But laws and policy aren't about feelings; they rely on consistent and precise definitions of terms.

If anything here is a strawman, it's your insistence on equivocating two behaviors which are substantively and observably different in their intentions, methods, and effects, in order to apply the moral censure earned by one disingenuously to the other.

If you want to make a meaningful argument against copyright infringement in its own right, please do so; I'd welcome the productive discourse. But arguing against it by calling it calling it by the name of another thing entirely - without bothering to establish a coherent connection between the two - doesn't constitute a valid argument in the slightest.

Further, my previous comment involved nothing resembling a strawman at all; you offered the position that copyright ought to be protected in order to ensure that the time and labor input into the initial design of a creative work would always yield a return for the creator, and I replied by pointing out that these are capital expenditures, to which no one in any field is entitled a guaranteed return at all.

Do you seriously regard as a strawman my classification of the advance investments necessary to open a factory and the advance investments necessary to produce a film both as capital expenditures, but simultaneously claim that your equivocation of copyright infringement with stealing is a robust and substantive assertion?

> I see no such distinction from a layman's moral standpoint

Then why do so many "laymen" actively assert this distinction?

Again, I have to point out that your methods of discourse actively reduce the credibility of your position.
Correct, this matches my observations as well. I'd trade an increase in mean latency for a decrease in worst-case latency anytime. It makes it so much easier to reason about how many resources are needed for a given workload when your latency is bounded.
We'd have one less rendering engine.
Makes that 30% Apple take seem a little more reasonable, doesn't it?
Intelligent routing would presumably need to know and act on a lot of state across their cluster, and if that's got to flow through a single node, you can see how it would present a bottle-neck as the cluster size and requests per second increased.

On the other hand, you can do randomised routing without knowing any state at all. You can do it with more than one routing node as well, which makes scaling almost trivial.

I presume there are Hard Problems associated with partitioning a Heroku-style cluster for intelligent routing, or that's what they would have done.
OK, speculation time.

My view: Valve is retreating from building the Steambox in-house.

Gamasutra mentions that the layouts might have affected the Android and Hardware divisions the most. The Valve Employee Handbook states that the ideal employee is t-shaped, which, I assume, might not fit the profile of someone hired to do Steambox engineering and prototyping.

Exciting times. I really didn't expect Valve to do something like this. Will be following developments closely.
Thanks for the write up. I've been looking for some more reviews on Heorku's platform and this in-depth review definitely illuminates some challenges with the platform.
How I have been doing it for the last year.

Puma define 4:8 threads or Unicorn 3 workers.
Thanks for the blog post, by the way. When we were struggling with our own Heroku scaling issues last year (we eventually moved to AWS), I came across it and it was good vindication that somebody else was facing the same issue.
If everyone agrees the person was less qualified, probably yes. But layoffs often hit people who at least one other person you want to keep thought was good, which hits morale, which in turn also drags down people. Partly it's because the web of connections is complex, and there are lots of reasons one of the remaining employees might have respected the laid-off employee, any one of them enough to instill a bit of a grudge against your decision-making (or even incredulity that you fired the now-fired person before them).

In a way it's the worst with well-run teams that gel. The ideal situation is that everyone works out what they're good at and develop a bit of a rapport. But then if you fire one of the team members, everyone else on the team has a quite first-hand reason to believe the now-fired person was valuable.
I expect her to land on her feet, I keep wondering about Abrash, is he still there? See http://news.ycombinator.com/item?id=3838880 which was about 9 months ago.

Given that Abrash is (was?) working on something Occulus Rift like, I wonder if they just decided "Hey we can use O.R. and not bother with this home grown thing." ?
But you killed it every time - especially the times where The Verge and Engadget used liveblogging platforms that crapped out on them. :)

Thanks for your coverage. Hope I won't have to follow my Apple keynotes without them.

Then again, it looks like Apple are moving towards livestreaming them, which would require blogs to adjust the way they liveblog the events.

What are you guys going to do at this point? We're in short supply of good tech writing.
Because time and time again, disgruntled and often soon-to-be or recently-former employees sabotage or steal from their employers. It's an IT security best practice to disable logins immediately when someone is notified or gives notice of termination. Often during the first HR sit-down. It's cold, but at least remaining employees aren't put at risk.
Its not an issue of it not working -- it would be illegal. Illegal things can not happen in plain site in the US, and the airport is one of those places.

Everyone knows the TSA is a joke, these are people who are qualified to do very little else. The TSA procedures are just noise to make the racial profiling look less obvious.
err : A$1099 * 1.04 = US$1142
Statelessness. You don't have to remember where you've sent recent requests, which ones are still in process, or how long they've taken.
Is Valve doing poorly, as the article suggests? I thought Steam was incredibly profitable for them. Could this be more about canceling projects that weren't going anywhere?
New Google Now cards + widget unveiled
I am saying that the whole point is the theater.
Well, usually that's part of the severance arrangement: you get to represent yourself as employed for some amount of time.

You do want people out of the office once you deliver the news, because even if you fire people fairly, they're not going to be doing useful work. Otherwise, I agree 100%.

Many companies use PIPs instead of severance offers in order to build a "performance"-based case against an employee. (This is sometimes part of a layoff being dressed up as a performance-based cut, and sometimes not.) That works out horribly. It's actually cheaper, considering morale issues, to cut a 3-month severance than to put someone on a 2-month PIP, because most people turn toxic during the "walking dead" period of the PIP. What PIPs are about is externalized costs: the HR office claims it "saved money" on severance payments, but the team and manager have to deal with a walking dead employee for 2 months.
I didn't jump to any conclusion that they are doing this simply for the money.

The only conclusion I jumped to was that they ditched the routing they originally said they had (without telling anyone) and that their routing is worse than what you get as a default from passenger.
Is there a reason for using both HAproxy and Nginx?
160 miles a day is pretty extreme, sure, but there are plenty of conceivable circumstances where you would do it, especially when you factor in kids going to school and errands.

If you have a long enough commute, you are spending 80-90% of it on the freeway, traveling at 65-80mph. 120 miles at 75mph is only 1.6hr.

For example, I have a friend who lives in West, TX and commutes to north Austin for work. That's about 100 miles one way.
The reason is threefold: morale, liability, and inability to use the worker for any long-term planning.

Having a person who knows they will be fired roaming the halls, poorly motivated is an obvious risk for the morale of the remaining employees. And it just drags out the pain of separation, even if the separation is otherwise amicable.

Second, there is some risk, no matter how generous the severance package, that the employee will become disgruntled and desire to harm the company. Even if it's a 1 in 1000 chance, the damage that a disgruntled insider can wreak is enormous. Huge downside, very little upside.

Finally, if they stay, they are now effectively temps. They can't be used in anything but short-term work because you don't know when they'll get the magic offer and be out the door.

A good severance package (say 2 months of salary, plenty of buffer to find a new job) costs just as much as keeping them on for two months after a warning, but has none of the downside risk. The only thing you lose by doing this is the very marginal short-term work you could allocate to them in the interim, which they would be poorly motivated to execute to the top of their ability.
To which legislation in particular are you referring? I often see this claim on HN, but I do not think that most people here have any real understanding of the situation.

As an anecdote, I work on a medical information system for emergency rooms. The VA uses our system in some locations. Federal regulations are not at all onerous. Things like HIPAA are dealt with using a few really basic rules.
How is that a straw man? I was attacking his (now deleted) point of view using exaggerated sarcasm.
I really like Rap Genius, but I wish they would tone down the blackness of the background. Reading #CCC text on #000 background makes my eyes bug out.
> I wanted to understand the mechanics of every lapse...

It's a futile exercise, and counterproductive. Therapy falls apart when you get stuck in the loop of asking "why did I do this?"

The important part is not to do that, but that's not what we want, because that would involve real, actual change. And as we know, feeling bad about yourself > the pain of changing.

> a girl asked me, "So do you prefer Vancouver food or Hong Kong food?". And I responded with a 5 minute dissection of the question to better understand what the question was asking, because the answer could vary depending on the question's true meaning and intent.

Your answer should have been, "howcome?", or "X is my favorite, howcome?"
I found the O'Reilly/CodeSchool "Try R" tool to learn R the best because you learn it by using it in a interactive manner http://tryr.codeschool.com/
Can you give more info on the soldiers attacking his lab? I have only found references to a raid on his home and lab by the DEA and local LEOs.
Ask HN: Does anyone need any Product Design help?
Hi Everyone,

I wanted to know if anyone needs any Product Design (UX/workflows/HTML/CSS/etc prototyping) help.

There's 2 reasons why I'm asking:

1. I have some extra time and want more experience doing Product Design.

2. I will be able to help you for free but in exchange I'd like to include whatever work I do for you in my portfolio.

Time: I can commit a couple of hours (3-4) on the weekend.

Let me know if you or anyone is interested. As always we will have to talk first and make sure this is something both of us want to do.

My email is rishi@rishtal.com.

Have a good day!

Thanks,

Rishi
If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.
We did this, but all it did was buy us a bit of extra time before we ran into the same problem again - a very small percentage (<0.1%) of requests creating a queue that destroyed performance for the rest of them. Also, FWIW, Heroku does not officially support Unicorn, and you have to make sure that you don't run out of memory on your dynos (we tanked our app the first time we tried Unicorn with 4 processes).
It's a way to point out how wasteful spending on the TSA has become. People lack intuition for numbers on the scale we are talking about. Does $8.71 billion seem huge compared to $8.70 billion? (I'd sure like to be making $0.01 Billion a year!) By suggesting something specific be done with the funds one can give some impression of the potential that is being wasted.

For the record, I'm not disputing the fact that airport security is ncessary. I'm just arguing that it's only good in moderation. The TSA is both too intrusive for the security they provide and extremely wasteful. Back-scatter X-ray machines are a great example of this. The first models used were not adequately tested and proved to be easily defeated. The solution? Replace them, at tremendous expense, with a new generation of machines that have also not been adequately tested. At the same time, x-ray back-scatter vans are being rolled out on a truly alarming scale. It's a cash bonanza for someone!
There's tremendous value in the history of a design file. Virtually every design agency I've worked for has had the need to write case studies based on work they've done. "Showing your workings" is also a powerful tool for demonstrating to clients the value for money they're getting. Finally, I can imagine such history being an invaluable aid for junior designers trying to learn from their seniors; it potentially transforms a nebulous process that is very hard to inspect (except by direct observation) into one that can be widely examined by distributed users.

What I'm unclear on is how the nitty gritty of the versioning actually operates. For example, most designers would attest to heading down many "dead ends" that may only ever live in RAM. Can services like this snapshot those interim states, or do they only capture changes to files on disc? Some experienced designers will also likely do their own primitive versioning by saving a design out to different paths as they go (home_v1.psd, home_v2.psd...). Can Pixelapse et al handle this user behaviour, or does it require people to change their workflow? If the latter, is an analogous feature available to allow designers to mark major version increments?

In many ways it ultimately feels that "files in folders" is a broken way to handle designs, units of design and design projects on a computer. I applaud attempts to supersede it. As long as Photoshop thinks in terms of files though, it will be a difficult space to innovate.
That's exactly what happened to us - switching to unicorn bought us a little time and a bit of performance, but we hit the exact same problems again after a couple more weeks of growth.
I'm not as happy with mine feature-wise but glad I participated. Build quality-wise it is pretty bad. I've already lost a button and reported it to them. I think of it as taking a cold shower before jumping in the iWatch hot tub.
As long as they've got a limit on the maximum amount of concurrent requests, they'll be affected. It might well not be as obvious.
I think he means he doesn't want to work in Iran, but is open to job opportunities outside Iran.

I am not very sure, but i think you also can't receive money when you freelance in Odesk/Elance if you live in Iran.
People should be free to take whatever soft drugs they like.
We will not win the war against drugs with prohibition, just
look at when alcohol was forbidden in United states during the 30ies that led to the rise of Al Capone and similar mafia organizations.

The solution is simple make it unprofitable for criminal originations to manufacture drugs by making them legal.

The current drug laws are also hypocrazy as long range bomber fighters are allowed to take light doses of Amphetamine to stay awake during long missions. So the state are allowed todo what its citizens are not allowed todo.
I wonder if he would ever have done this if SSDs had arrived a few years earlier? In 2007 my hard disk was indeed the greatest bottleneck on my computer by an order of magnitude. Replacing my spinning platter with an SSD (around 2010) has all but eliminated this problem.

It's funny how innovation works: the advantages of a Chromebook are no longer linked to it's original purpose, but we have found the incidental advantages that came along with solving the original purpose so beneficial that it's now carving out a solid niche based on those alone. It makes me think of a detonator and an explosive. You need something to initiate an innovation, but that doesn't always have to be what sustains it.
Cute, but I wonder if this is legal or not, or if PG would enjoy someone else profiting from his stuff.
Do not use these companies:

Chase bank or American Express

They will turn there back on you when you need them most. I had a horrible experience with both back in 2008 when the economy was collapsing. And no, I don't have crappy credit.
Re the distribution, absolutely. That "FIFTY TIMES" is totally due to the width of the distribution. Although, you know, even if their app was written such that every single request took exactly 100ms of dyno time, this random routing would create the problem all over again, to some degree.

As for the intelligent routing, could you explain the problem? The goal isn't to predict which request will take a long time, the goal is to not give more work to dynos that already have work. Remember that in the "intelligent" model it's okay to have requests spend a little time in the global queue, a few ms mean across all requests, even when there are free dynos.

Isn't it as simple as just having the dynos pull jobs from the queue? The dynos waste a little time idle-spinning until the central queue hands them their next job, but that tax would be pretty small, right? Factor of two, tops? (Supposing that the time for the dyno-initiated give-me-work request is equal to the mean handling time of a request.) And if your central queue can only handle distributing to say 100 dynos, I can think of relatively simple workarounds that add another 10ms of lag every factor-of-100 growth, which would be a hell of a lot better than this naive routing.

What am I missing?
My company switched off of Heroku for our high-load app because of these same problems, but I still really like Heroku for apps with smaller loads, or ones in which I'm will to let a very small percentage of requests take a long time.
Repost?

This is an attempt at dry humor. Or more accurately in this case, dark humor.
The Jeopardy Teen Tournament Just. Got. Real.
I've known quite a few people who take mushrooms as party drugs, as well. Anything is a party drug to the right group of people.
> "I wasn't even afforded the privacy of a screen."

this is... interesting. I mean, I know this is very different for me because of the cultural expectations of me as a male, but personally? I'd /much/ rather have an authority figure violate me in public than in private. In public, they are restrained by witnesses; in private? well, who knows what they will do? and if I complain of what they do in private, well, it's my word against that of an authority figure. In public, there are witnesses.
Except that Google takes the same cut ...
Yeah, the only real question is whether or not it's true that they no longer do intelligent routing. If that is the case, then regardless of anything else the problem exists once you pass a certain scale/request cost. It won't matter if that one dyno can handle hundreds of requests at once, it will still queue stupidly.
That's probably true, but the value that Heroku is selling (and they charge a lot for it!) is that you _don't_ need to deal with this - that they will balance that out for you.
Could someone with an appropriate background analyze the license in-depth?

Looks like it allows you to produce derivative products, which is great, however, it also forbids commercial use, meaning it's at least not at the same level as FSF- and OSI-approved licenses.
Nevertheless, random routing is a bad idea even if a dyno can handle multiple requests simultaneously
DBMS pagination on Oracle and MySql
Yes, it's true that the Cedar stack supports forking web servers like unicorn, and that an individual dyno can run multiple workers and therefore serve multiple requests at the same time.

However, dumb routing is still very problematic – even if your dyno can work on two requests simultaneously it's still bad for it to get sent a third request when there are other open dynos.

Also, for apps with a large-ish memory footprint, you can't run very many workers. A heroku dyno has 512mb memory, so if your app has a 250mb footprint, then you can basically only have two workers.

Another essential point to note is that the routing between cedar and bamboo is essentially unchanged. They simply changed the type of apps you can run.
>The party implied is democratic liberalism, which opposes the perceived nepotism in government due to a saturation of rich individuals and leverage afforded by wealth.

Is it? "The government is corrupt and incompetent so they should stay out of it" is a basic conservative/libertarian position.

I think a lot more people would agree with conservatives in principle if they weren't so hypocritical and corrupt in practice. You hear the occasional lip service regarding the poor or power imbalances reinforced by government policies, but they never seriously advance policies that would do anything about it, and any time they actually propose or enact anything specific the result is usually the complete opposite of that. Cue the old saw about Republicans believing the government is corrupt and incompetent and proving it every time they're elected.
This is obviously a way for the bay to point out the absurdity of copyright law.
Yes, because we're ostensibly mature adults.