skysight-inc/HackerNews huggingface dataset:
Almost every place I've lived in the US construction does occur 24 hours, but maybe not 7. Commute and daily traffic require it. Maybe you're just not out 24/7 to see it?

Most traffic systems are automated and their goal is to MOVE TRAFFIC THROUGH. The minimal use sensors in the roads to help determine when to switch lights. The most advanced are integrated and use many metrics and algorithms in controlling traffic signals http://trafficinfo.lacity.org/html/atsac_1.html
:)

Benchmarks, especially those from card manufactures are suspect. Just because you've got a network ping-pong test running at essentially line speed, doesn't mean I'll see any of that. I fully agree.

Filling any pipe with your 'rsh' benchmark is harder - on all the Linuxes I have on hand, rsh is aliased to ssh, and I've never seen a system deal with the overhead of ssh adequately. (There's high perf ssh if you really want; I've never tried it. http://www.psc.edu/networking/projects/hpn-ssh/) (Would you accept numbers over netcat?)

If I'm not getting 90MBytes/sec over Gigabit, then something, somewhere, is slowing it down.

We sell servers that sustain 300+ MBytes/s per client, via Samba for two clients. Those clients are very fussy about latency (video editing/DI). That's what we sell, so it's empirically that fast; ie not a benchmark. The server itself will actually do 900+ MBytes a second (to three clients, the same as page 14 on your linked pdf), but my small print is the market doesn't want that configuration.

These numbers are what I know, which doesn't include NFS. NFS-RDMA and pNFS are supposedly good for performance, but I don't have any first hand knowledge.

None of what I just stated contradicts you. But if I'm at 900 MBytes/sec with my level of hardware, I find it hard to believe someone with more budget isn't capable of a full Gigabyte/sec over Infiniband, arguably HIPPI's successor.

I, too, lament the demise of SGI.
My impression is that pointing out a typo doesn't further the conversation unless the typo completely alters the meaning of the comment.
"Pound" as in "Pounds Sterling" not "Pound" as in "Oglethorpe" I take it?
I think your spelling correction was downvoted because it contained a spelling error and because you came off rather paternalizing.

Humor in general is downvoted on HN, especially one liners unless they're really good.

Don't take it personal.
U.S. Teachers Get $442 Million Pay Raise
FarmVille.
No, this is not correct. The idea of sendfile() is that you can send a physical address (probably something in the buffer cache) directly to a network adapter in a zero copy manner. That's current impossible with a send()/write() to a file descriptor in Linux because you can't construct an SKB from a userspace address without forcefully pinning memory. Pinning memory from userspace is a privileged operation. OTOH, since buffer cache is not part of the memory of a process, you can obtain a physical address of it.

There's no magic in the kernel that avoids context switches. If a kernel thread has to switch to another kernel thread, that's still a context switch. And syscalls are ridiculously cheap on modern hardware. sysenter is like < 100 cycles.

That said, there's no theoretical advantage to doing this via sendfile verses doing a send() from an mmap()'d file. If Linux was smarter, it could realize instantly that the address given to send is file backed and could construct an SKB from the physical memory without pinning.

This is what the reference to a "5 minute hack" was vs. Rob Pike's claim that the interface exists to work around a problem in Linux. He's completely correct.
With sensitivities that low false positives are triggered by cars in other lanes. Induction loops are very sensitive to variations. Believe me, I spent many hours on tweaking detectors.
Thanks Jay, I love to have open dialog.

Rypple itself may be young, but the guys who co-founded it are very experienced in the world of traditional, capital-E, Enterprise software. They were the head of sales and marketing at Workbrain, which they grew to $100m in revenue providing serious Enterprise solutions to the biggest companies in the world. They intimately understand how this world works and so they are well qualified to say that our approach is a much better one.

I didn't mean to insinuate that the companies are somehow unqualified to make those judgements. It's usually the armchair experts that make these kind of comments. When I referred to the young age of the company I was really referring to the company model itself instead of the people inside. I wish you the best of luck but my point was mainly that knowledge and acumen are great, but when it gets down to it the proof is in the pudding. I hope you guys will still be here in 10-15 years and it will be great if you are and the model does scale into the long term, but I think it's premature to say it's definitely succeeded before the eggs have hatched.

The strongest reaction to this post is around the idea of removing features, with lots of people saying "You can't do that!" and "Your customers will revolt!". We get that. It's the natural reaction after years of building software in the traditional way. Here's the thing though: our customers celebrate when we take features out. We get emails from them thanking us for keeping the app simple and jettisoning the cruft. We sometimes pull something out that we do get a strong reaction to, and then we work closely with the users who felt strongly to make the feature awesome and put it back in.

I think this really depends. Mainly it depends on what kinds of features you're removing. One thing I'd really caution against is changing your API every couple months. That is the kind of thing that can drive your company into a hole they can't crawl out of.
Stocks are (usually) bought based on future earnings potential. So for an oil company, whilst oil is needed, it is unlikely to suddenly make a huge profit over the other oil companies and instead will provide relatively safe long term returns.

So its price reflects this both good and bad stability.

On the other hand Apple has a track record of coming up with almost totally new markets which it can dominate and take the lions share of the profit, or find a niche to exploit in an existing market - ala iPhone. So it's price is speculating that they can keep finding these huge new profits in addition to the huge profits they already make.

tl;dr Apple is like a giant profit finding machine.
Now, now. Let's be reasonable. No one is valuing Facebook at $866.67 billion.

/pedantry
This certainly won't be the first time that a system designed to increase uptime actually reduces it. I've seen a lot of "redundant" systems that are actually less reliable than simple standalones thanks to all the extra complexity of clustering.

I guess at Facebook's scale you have to build in fallbacks but this is a reminder that you can easily do more harm than good.
There's a special place reserved in Hell for the management of Borland, right next to the management of Commodore

A large part of their demise was due to their constantly pursuing the enterprise market (they even changed their name to Inprise for a few years) where they were largely ignored and unwanted, while ignoring the indie devs and smaller shops who loved them.

In the end their developer tools went from best in class (by a very large margin) to an embarrassing copy of Visual Studio, but not quite as good and a hell of a lot more expensive.

Kylix was a bold move though, and could have been something good (it wasn't). For anyone interested, Lazarus is still going; it's a cross platform open source ide heavily inspired by Delphi (version <= 7) http://sourceforge.net/projects/lazarus/files/
Google Gets Chrome Web Store Ready To Roll
I'm assuming that the real objection to an interface like sendfile() is that you shouldn't need to copy_from_user() just to construct an SKB if the user address being referred to is pointing at a file backed VMA. If we could do a zero-copy send() from userspace, there would be no need for sendfile() in the first place.

IOW, { data = mmap(fd, ...); ...; send(fd, data + offset, length); } could be just as fast as sendfile() if we were smarter.

We're starting to build infrastructure to do zero-copy networking in userspace specifically for KVM. So far, it's not at the socket level but instead at the macvtap level but that's arguably another problem with the networking stack--macvtap/tap should just be an address family :-)
> It's never been very clear what "open" means, but I'm pretty sure it doesn't mean "untrained persons can perform major surgery risk-free".

With Android, "Open" means you must perform major risky surgery on your device to install software that hasn't been approved by your carrier.

With iPhone, "Closed" means you can always install the latest OS as soon as it's released.
Blogger and witness are two entirely different roles.

Witness: I can to the lunch uninvited, people squirmed, a couple of the participants later told me they were uncomfortable with the tenor of the discussions. (From a witness perspective, first-hand facts based on personal knowledge are everything - anything beyond this is hearsay and not to be credited).

Blogger: These are the top angel investors in the Valley; I heard second-hand from a couple of the participants that they are upset about valuations, are looking for ways to cooperate to bring them down, and are also looking for ways to cooperate to freeze out VCs as co-investors and to limit the impact of rogue operations like Y Combinator; I checked with my lawyers (and am myself a lawyer) and I can tell you that this stuff, if true, is blatantly illegal and I speculate that this is why everyone was squirming.

This latter stuff is the blogger, not the witness, drawing inferences, putting glosses on the limited known facts, speculating, etc. with an overriding goal of making a story significant, newsworthy, and even sensational. This is a legitimate role of a blogger, and is a big part of what makes TC and Mr. Arrington himself sometimes insightful, sometimes arrogant, sometimes maddenlingly infuriating, but almost always interesting to a general readership that takes an intense interest in the subjects being covered.

One can "trust" the witness aspects without necessarily giving credence to the blogger glosses that are put on the facts. Mr. Arrington appears to be a straight arrow when recounting facts but watch out for those glosses: they might easily have an agenda behind them as to which the simple facts themselves are secondary (this doesn't mean they can't be true, only that they need to be considered cautiously in light of possible underlying motives).
That was an excellent description of the problem. Too excellent, it seems, for many of the commenters. :-p
I know. I just hesitate because "memory" in my field almost always refers to RAM.
The biggest domestic policy challenge in one graph (OK, two)
Of course; that's how it is now. Car rolls up behind you and you're fine.

The problem is when you're alone and there are no other cars in your lane. If you're totally alone, you can run the red light, if there is other traffic just not in your lane you're screwed.
The law isn't black & white either (despite what engineers would like to believe).
No. If the act of write() is to construct and transmit and SKB, and SKB can be created from a scatter/gather list of pages plus an offset into the first page.

But this implies that you can track the SKBs lifecycle in the kernel such that you can know when the SKB is done being used and therefore can allow write() to return.

The trouble is, a write() to a socket doesn't actually generate an SKB. Instead it copies data to a socket buffer and then the socket buffers are used to generate SKBs. Some of this is meant as an optimization to create large packets even from small writes.

But of course, a copy is a copy and sometimes this buffering is not actually helpful to userspace.
I say this coming from a practical point of view though. If I can sell something today for at most 15 bucks, that is how much it is worth, no?
> stories seem to stay on the front page for 2-3 days sometimes

That is far too long. I'd like the front page to churn twice a day so it was all new in the "morning" and all new in the "afternoon".
My favorite comments on fb page:

Stick with Mysql

* * * * If it aint broke, dont fix it!

Me too Melissa, and it's out there in the media that a group of hackers caused the problem, is this true, Mr Robert Johnson?

PLEASE !!!! WHAT CAN YOU DO TO HELP THIS FROM EVER HAPPENING AGAIN???????????????? PLEASE!!!!!!!!!!!!!!! CANDY

Kip da updates comin'

Did anyone get a message like I did about someone trying to access your account from another state?
> His conclusions are premature.

I think he even says he consulted a lawyer. I'm not sure what else you want, short of the feds getting involved..
I dont disagree with anything you or dhh said although I don't know for sure if his predictions will hold.
This is known generally as the "Thundering Herd" problem:

The thundering herd problem occurs when a large number of processes waiting for an event are awoken when that event occurs, but only one process is able to proceed at a time. After the processes wake up, they all demand the resource and a decision must be made as to which process can continue. After the decision is made the remaining processes are put back to sleep, only to wake up again to request access to the resource.

This occurs repeatedly, until there are no more processes to be woken up. Because all the processes use system resources upon waking, it is more efficient if only one process is woken up at a time.

This may render the computer unusable, but it can also be used as a technique if there is no other way to decide which process should continue (for example when programming with semaphores).

Though the phrase is mostly used in computer science, it could be an abstraction of the observation seen when cattle are released from a shed or when wildebeest are crossing the Mara River. In both instances, the movement is suboptimal.

From: http://en.wikipedia.org/wiki/Thundering_herd_problem
Democracy is an abuse of statistics
Agreed. We are valuing the whole pie based on the cost of a slice, which is problematic for many reasons...
You should look at the Wesabe code that got open sourced. They wrote their own scraper for a huge number of banks. That could solve your problem at the beginning. Although, it also creates a new problem for you, securely storing bank credentials. They open sourced a solution for that too, but I'm sure it'll still create costs for you.
Unfortunately your short post came over as somewhat trollish.
Hang in there, it will be worthwhile!

I too had similar problems getting a hang of Python. "Dive into Python" was a help. But once it 'clicked' it became hard to write anything that didn't work. One thing that I still marvel is at, that you can often just make a guess and it works! The collection of 'batteries' is very useful. Typing help(some_function_or_class) at the command prompt is often very useful.

NB: if you are writing for Windows and Linux and OS/X, then you should use the functions in os.path module to avoid hard coding the directory and filename conventions.
Yeah, I don't know how regular users find posts like this... but that was way over everyones head.
The pipe intermediary doesn't necessarily add a copy. You SPLICE_F_MOVE the data from the first fd to the pipe, then SPLICE_F_MOVE it from the pipe to the second fd. If either or both of those can be done zero-copy, they will be. The pipe intermediary is just a way of holding onto the reference-counted pages.
Agree, ideally they should of made two posts, one like this, and another for the 'normal' human being.
Intercal. Abandon all sanity, ye who enter here.
http://www.catb.org/esr/intercal/stross.html
http://www.muppetlabs.com/~breadbox/intercal-man/
For a second there, while reading the comments, I wasn't entirely sure I hadn't accidentally been redirected to a redesigned YouTube.

Interesting article, terrible comments.
Unfortunately its difficult to avoid relying on other businesses for crucial systems.

Example: Payment Processing, try building a business, especially an online one, without depending on Paypal, or at least Mastercard/Visa.

Sure you can reduce it by say not having FB Connect as the only log-on option but the simple fact is that you will almost always have to depend on other businesses, the best you can do is diversify and hope for the best.
"...feel bad that my comments were seen as problematic.."

Refreshing attitude for a message board. You should stay on, keep posting.
If any Canadian startups are reading, my venture is http://SmallPayroll.ca. Right now it's targeted to a slightly different audience, but I'd love to hear about what it would need to appeal to the startup crowd.
They built their own custom DNS server? Most of the failures that people encountered were a failure to contact the nameserver itself. Perhaps in the rush to try to fix it someone screwed up that as well.
I'm often encountering systems that are designed for very short queries processing small numbers of rows where the number of connections from the app servers is configured to be far greater than the number of CPUs.

Since those systems are doing very little IO, configuring connection pools to start with much more connections than the DB has CPUs and to add more if the connections are busy (i.e. DB is getting slower, probably because it is highly loaded), is guaranteed to cause resource contention on the DB and escalate the issue in case something goes wrong.

Its a total waste and yet the most common configuration in the world.
Sometimes cluster systems are mistaken for high-availability solutions while they are actually load-balancing solutions and can decrease availability due to added complexity and dependencies.
I wish I could upvote that more. PLAI was awesome. To me, scheme is to lisp what coffeescript is to javascript. More or less. Don't let it intimidate you. Throughout the book, you'll see awesome things, like how switching from static to dynamic scope is merely changing a bunch of lines of code. When I finally got my type inferencer working, I was euphoric.
Leaving Microsoft
Let me put it this way: if this story came from the Washington Post, New York Times or Wall Street Journal it would a) be more measured and b) have a higher standard for making what is a serious accusation.
I agree this write up is a bit silly but if we want to define an open platform it should probably be as open as x86 PCs. Who would buy a PC that locked you out of the BIOS and only booted a digitally signed / approved open source OS? That's basically what most of these Android phones are offering. I don't think that's good enough.
Sandboxing - OS Security Technique | ICA
The link you cited shows total college student enrolment increasing from ~15 mil to ~18 mil from 2000 to 2007.
Yeah, I noticed that too, it's kind of sad when you think about it
This is how http://twitter.com/facebook (how ironic)
Ironically, you were a bit too terse for me to understand your point. Would you care to elaborate please?
Repost here:

"But in 2005, during a major overhaul of the bankruptcy code, private student loans were given an elevated status and thus couldn't be discharged. This didn't make sense. If we are going to have a fair bankruptcy system, private education loans should be treated the same as other private consumer debt. That's the risk lenders take, similar to the risk borne by providers of loans for cars, homes or other consumer purchases." [1]

"Lenders receive a 97 percent guarantee against default losses, which removes almost all of the default risk of the loan to the lender. If a borrower does not repay his or her federal student loan, the government pays the lender 97 percent of the outstanding principal and all of the accrued unpaid interest. In other words, the lender assumes default risk for only 3 percent of the loan principal." [2]

[1] http://edlabor.house.gov/edlabor_dem/mt-search.cgi?IncludeBl...

[2] http://febp.newamerica.net/background-analysis/federal-stude...
So… you're new to Ruby
Yes, and in those cases it isn't about the drinks and the conversation isn't really casual.

What do you call ten competitors doing business development together?
I guess it was the At&t Samsung Captivate
How about Agda? It has the interesting property that it's runtime-error proof.

http://wiki.portal.chalmers.se/agda/pmwiki.php?n=ReferenceMa...
I disagree. I moved here from Indiana for a job. The company flew me over here for the interview and also paid for relocation when I moved here. Same situation for all my friends. We are new college grads, so the situation might be different.
"This means that Facebook is a database-driven app :D LOL , just refactor FB, you know it is a mess."
Very true - the complexity of the system also comes into the equation when things go wrong and it takes real people to figure out why an outage is happening. Highly complex systems imply longer debugging time, and at a certain point a theoretically lower up time can give you higher practical up time just because engineers can actually understand and debug it.
Have you looked at Heron http://www.heron-language.com/
Well if complex syntax is your thing, then don't go there.

Lisp is a lot more exotic than you might think! What it does to your brain is unmatched!
Have you tried a question mark at the start of your query? (i.e. ?google maps api)
As well dyslexia, is a common problem among right brained people there are a good deal of right brained designers and developers. dyslexia is something that they cannot get rid of, they can work to improve it but it cannot be "cured". Highlighting it can be taken as being an ass to someone with a disability. The problem is in all other matters they are intelligent so many are extremely embarrassed of their inability to spell or put sentences together in proper order as they know individuals will use it to detract from their message. To knit pick their spelling can be misconstrued as an attempt to discredit their argument based on a technicality.
Is it a legitimate goal of a blogger? What do we, as the audience, get out of it other than a hazy pseudo-reality? I see what MA gets out of it...
Okay, well in that case, I am also ... (takes comparably deep breath) ... going to do this. And I will attempt to create a game that is even more stupid-simple than yours.
The Way We Live Now - Home-Schooling for the Techno-Literate
I have an Greasemonkey script called HN Toolkit that lets you display the new post & top post side-by-side. It really helps you catch the good ones that aren't upvoted.
Listened to a lecture by Rob Pike last week. Blown away!

This guy has consolidating what he and his cohorts have learnt from C, C++, ANSI C (Plan9), Limbo (Inferno).

Might be the most cutting edge language of the moment.
I dig the UI. Very clean.
Get The Best Spy Gadgets and Spy Gear Here
Spy gadgets is a collection of "spy equipment" we have found for sale around the internet. And many more real security surveillance equipment and spy gear are available at 007spygear.
Several points:

1) Yes, these values were created long ago The question is if they still apply.

2) While being a single-mother isn't nearly as destructive these days, there do seem to be significant benefits to the child from having two parents. (how strongly is up for debate -- admittedly there are conflating variables here).

3) The father not taking care of the kid also has deleterious effects on the mother's career.

4) Even from a selfish perspective, following such advise reduces economic burden from child support. And social burden since society generally has such a value (rightly or wrongly).

5) Birth control is reliable enough compared to other risks in life. See http://en.wikipedia.org/wiki/Comparison_of_birth_control_met.... Pick 2 independent methods and use correctly. 10 year failure rate is on the order of 1 in 300 (only triple that of dying in a car accident). Add a third and odds drop to 1 in 10000 (car accident death order of magnitude higher).
Perhaps the DNS issue was their way of shutting off the site.