skysight-inc/HackerNews huggingface dataset:
I'm a huge fan of both Nginx and HAProxy - used together.

[internet] <-> [Nginx] <-> [HAProxy] <-> [app servers]

Nginx is a great webserver, but isn't a good load balancer. You can install a patch that improves the balancer - http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer... - but it still isn't as nice as HAProxy

With HAProxy the status of the system is visible. For the largest site I use HAProxy on I keep my status page public - http://userscripts.org/haproxy - but it isn't required.

HAProxy is particularly good for rails since you can say each app server can only have 1 request at any time. This makes requests queue at the HAProxy layer, so if an app server has a request that takes a extra long time you don't have requests waiting for that app server to finish - instead you wait for the next available app server in a FIFO queue.

Combining HAProxy with munin gives great stats for tuning your system - whereas just nginx with the patch had no visibility into where bottlenecks might be.

I
Ajax performance analysis
I'm curious where you get the "MySpace is about 4 times bigger" number? My understanding was that Facebook had grown to be larger than MySpace in some foreign markets, and that it was about half the size in the US. It is still firmly in second place--MySpace usage is still growing roughly on pace with Facebook so Zeno gets his revenge--but certainly more than 25% as popular as MySpace.

I'm not saying Facebook is worth 15 billion (that'd be stupid...I'm talking Stave Ballmer level stupid), but one can't say they've lost to MySpace, when the market is still wide open worldwide, and Facebook seems to know how to execute internationally than MySpace.

But, I think Facebook will lose, in the end, to a few dozen companies that build a better garden without the walls. Each will be smaller than Facebook, but they'll end the reign of Facebook and MySpace with a wholly different process of social networking. (I think a single destination "social portal" is as doomed to failure as the whole web 1.0 "portal" idea turned out to be...portals will always be less interesting than the whole Internet, so when the whole Internet is socially enabled the social portals will falter. The oracle has spoken.)
That Guido got to work on a version of Mondrian for the outside world indicates less about Google's willingness to open up and more about their need to please their superstar.
It doesn't have scoble attached to it.
IPhone Safari doesn’t remember passwords, yep I know - rovivor.com
Interesting example indeed. I'd say they are a perfect illustration of a feature that, in the words of the article author, "[tries] to abstract out some behavior that is not that hard to deal with anyway".
Consider also using multiple DNS A records - the selection would be random thus balancing the load.
For example, do "nslookup google.com"
We're hosted at EngineYard, which use nginx with a fair load balancer plugin that ensures that new requests are assigned to free mongrel instances (yeah, it's RoR).

Works great.

Daniel
In fact, you hit the nail on the head. They're discussing canning the project but appreciate I've put heart and soul into it, and are looking at all the options about how to go forward. Fortunately, one of those options includes keeping me on, so they're still paying me. This in itself is one of the lessons I've learned: I don't have to be perfect to be useful. Believe me, that's a real paradigm shift.
You guys get show and tell sessions throughout school, don't you? In all of school until university I gave a grand total of 1 presentation to my peers. Then one more at university.

Daniel
So I think the jury is still out on Jonathan Schwartz, even with yesterdays disappointing earnings statement. The previous three quarters were profitable for Sun (which was a pretty amazing turnaround--Sun was in deep trouble before Schwartz took the helm).

The thing that a lot of folks haven't realized is that the nerds have recaptured the flag within Sun. And, you know what happens when a bunch of nerds are on a mission. That's right: http://tinyurl.com/67user

OK, so maybe putting the nerds in charge isn't a guarantee of success, but it is a prerequisite in a tech company.

Everybody can probably think of one cool thing Sun is doing, because it touches on your specific area...but when you do an aerial survey of the whole Sun "cool stuff" landscape, it looks really impressive:

Zones - A sweet elastic computing solution. More efficient than Xen (due to different model...more like vservers on Linux, but it's actually stable enough to use), allows quite impressive run-time configuration like scaling resources up and down, etc.

JVM for dynamic languages - Sun has drunk the Koolaid on dynamic languages, and is actively contributed to a bunch of them. I think they are beginning to understand that Java is not the important thing to come out of Sun, but the JVM and that huge library is.

Cheap hardware - Really, you can buy Sun boxes for not too much more than equivalent Dell systems. And, Sun boxes are nice. They don't have as many low-end options as Dell, of course, but once you get into high end gear where Dell loses its huge quantity advantages, Sun is a viable option.

MySQL - Most popular database in the world. Hard to top that for importance in the industry. During web 1.0 they had to team up with Oracle to offer the whole solution...now they've got the full stack.

Caroline - Utility computing, elastic computing, "App Engine", call it what you will...Sun is building a commodity solution to the problem.

Will all of this make up for years of resting on their laurels? I dunno...but it's too early to count them out, and Schwartz seems to have played an important role in the change.
"The survey also revealed that a disproportionate number of people under the age of 30 and retirees in the group made $20,000 or less a year."

Perhaps because they're more likely to participate in a telephone poll of this kind?...
I use perlbal, which is nice-n-simple. Add a few lines describing where your backend servers are to the config file, save it, and start perlbal.

For our $WORK applications, we just have an apache that proxies to the backend FastCGI apps. We don't have a ton of load, so that works fine. (We might be switching to nginx, which is much simpler than Apache for this use case.)
Is he a VC/angel now?
Speaking of Caroline, are there any news? It seems to me this page was not updated in quite a while. http://research.sun.com/projects/caroline/
The ultimate goal of a warrior is to lay down his sword.
Let's put this in straight, un-spun English: gas in other countries is taxed heavily, and it is not in the US. Therefore, the US deserves for the multi-national oil conglomerates to artificially jack up prices.

Yeah, that's logical.
Schwartz has never mattered. Ever seen anything by him? It's astonishingly obvious that he has no idea how to make money. He does know how to use hip buzzwords, though.
Installing XCache From Repo and Source
Scoble's job seems to be to promote new startups. The article would have been much more interesting if after his whole "smell" thing, he ended up saying this one smells bad. But because its a positive article, one is left wondering why he couldn't actually come up with something substantive, that the best thing he could think of to say was that the company had a nice aroma.
There is also GNU Texmacs:

http://www.texmacs.org/

It's excellent, but from what I can tell it's neither TeX nor Emacs. An odd name, but good software nonetheless. (You can export to TeX if you want, but that's not its native format.)
Easier to use than typing something like:

Here is an equation: $y = x^{2x + 1}$.

Probably not. GUIs trade learning for speed. You never have to learn anything, but you'll never be fast. The GUI will always be in your way, slowing down your work. TeX is the opposite -- you have to read a lot to get started, but then the only difficult or tedious part of writing will be the subject matter you're writing about. The only barrier between your mind and beautiful output will be your slow fingers :)

LyX and Texmacs are a good compromise, however. You can write the code when you're comfortable doing so, but you can click stuff when you're learning.
After reading over it again, I wonder if he was misquoted. Because, if you take into account exchange rates, then the US probably is probably paying more on an absolute basis. But it doesn't say anything about exchange rates.
Living Room of the Future (From 1979)
I still maintain its easier to get your professor or colleagues to upgrade to Word 2007 than to convince them to learn LaTeX.
Anything related to technology is surprisingly accurate.

Anything realted to style or culture is hilarious (check out his outfit & the drink robot).
+1. Round robin DNS has worked well for me in the past,
ever heard of laser surgery?
There's a uWink coming soon to downtown Mountain View (corner of Castro and California where they old ranch house used to be). I just hope it isn't too gimmicky.
There's a risk associated with that.
Oh, right. I got it for free. :)
What's the point? Is this better than KVM?
what is your app server? or is this only http?
which is why we'll keep driving cars in the US until it makes sense not to, which will be several more dollars/ gallon.
So do you think NASDAQ:JAVA is a good buy right now or it is too risky?
why would you even waste your time building on google apps?
Don't you forfeit too much control this way? For example, the load on your servers would be determined by the caching behaviors of all the various DNS servers and clients out there. I think this scheme should only be used as a front line that leads to another layer of load balancers.
take two shots of expresso and get to work :). alternative would be to splash your face with ice code water and go out for a quick run.
do any intelectual thinking after.
Laziest straw-man argument I've read in a long time. Web 2.0 and/or cloud computing is not about the old "thin client" idea, in fact, it's more like the opposite (with "fat" clients being part of the cloud).

The writer is just trying to re-hash a very old debate nobody is having anymore. Ars Technica's standards seem to be dropping rapidly these days.
The Twitter Conspiracy
I'm using nginx in a couple different setups:

nginx -> paste (pylons)

nginx on one machine, three other machines with 8 instances of paste each. Nginx proxies directly to the paste port (which incidentally is also itself a threaded server, but I've gotten best results by running several instance per box). Volume has been as high as ~8mm dynamic requests/day.

nginx -> lighttpd -> php-fcgi

nginx on its own box proxying to 8 app servers each running 160 php-fcgi instances. Volume here is ~16mm dynamic requests/day.

Both have worked pretty well so far. As anortherjesse said, there's not a lot of feedback, but it's done everything I need so far.
Aside from the robot to bring me beer, this is one of the best prediction-sets I've seen. I shop online at Peapod for groceries, and I'm scanning some docs to email to my accountant, so the "mail-slot" is about right too. Although, looks like they're predicting this for 1989, so they were a little optimistic. The funny thing is that this seemed wild and futuristic then, and now it's just mundane.

I try and maintain my sense of wonder...
Oh, sure, I could use some support staff, but I could use hackers more.
The idea is that there is enough chaos in the Internet to produce practically random result with sufficiently large number of users.
Random is pretty good, unless you must proactively shift load to least-loaded systems (in which case you have to be super careful to avoid oscillations) .

One drawback is that you can't take system out of rotation as quickly, but this can be somewhat mitigated by setting very low TTL (60 seconds).
Scientific studies show that caffeine is most effective when consumed slowly over a period of time. So sip that double latte and make it last until lunch.
We use HAProxy because we're very session based (each user sees an entirely different thing) and HAproxy was a good choice for that. I wrote a post on setting it up a few weeks ago.

http://leavingcorporate.com/2008/03/03/session-based-load-ba...
In my experience, honesty is your friend here :) I had a contract that we took on which inherited someone else's codebase as a starting point, but it sounded simple enough.

When we realized it was getting not-so-simple we had to decide whether to go back to the client and explain that it's going to be more than we thought, or whether to just push through and go over a bit (false coder heroism...). We decided to push through. Turned out that just made things even worse and we would need even more time to fix it now.

We were also busy with another big project at the same time, so we were focused more on that as well. So things dragged on. Finally, I had to have a meeting with the client where I could make something up or simply tell them the truth. I opted for the truth (that would be my dad's influence :).

I was pretty scared it wouldn't work, but they accepted my apology and the plan to correct it, which did require more time and money to complete. I felt like I had dodged a bullet. We managed to sort everything out in the end, learned our lesson, and they've since recommended us to several other companies because they feel they can trust us. And it feels pretty good to have earned someone's trust that way too.

Anyway, I write way too much on here... (gotta learn to edit! ;) But good luck with your project, hopefully you'll have the same luck I did!
I like Pound very much: it's simple and robust.

However, I recently noticed a memory leak. We use healthchecks on our production machines, which means a consistent rate of hits every few seconds, 24/7. After about 3 months, Pound had chewed up 1.7GB of RAM, which caused memory usage alerts in our monitors.

Not a big deal -- you can always restart the process -- but I'm still evaluating alternatives.
Removing the "Intel" from the title is the only thing that made this title interesting to click on -- maybe someone has an interesting argument? But no, the entire article is Intel preferring more people buy bigger processors in greater numbers. This is not news.
Hello, sock puppet.
so i'm a VC and i don't bother to ask that question. there is no value in asking that and not a question you want to answer IMHO. Most important is to get referred into a VC if possible from someone who trusts you and who the VC knows well. VCs are a lot more frank with a trusted referrer usually than they are with the entrepreneur and the referrer can signal that the deal is getting traction. That seems to matter, probably too much, to most VCs.
I've been using Apache 2.2 with mod_proxy_balancer to do load balancing for PHP and Rails apps for over a year now, and had pretty good experiences with it. Since I support a lot of existing Apache servers, the configuration is easy for me to work with, and the ability to do authn/authz and SSL termination at the load balancer lets me keep the load down on my application servers.

It's probably not as fast as Nginx, but I haven't found our load balancers to be a bottleneck. In fact, we've been doing load balancing on a pair of really, really wimpy servers (Celeron CPU and 512MB of RAM) running Apache on OpenBSD for about a half-dozen different apps for the last year, and never seen the average load climb up over about 0.5, even while handling upwards of 300 requests per second.
Mechanical Turk is cool. Anyone used it for something interesting?
I'm using it to have people find proper citations for scientific papers based on press releases. It works really well.

Has anyone here used Mechanical Turk for any interesting projects?
All polls are like that. The industry pretends that they are dealing with random samples, but they are never, ever doing so. They select for people who, unlike me, don't hang up the instant they recognize a cold call. I've got it down to 2, 3 seconds top...click.

One sure sign: A slight pause between my "hello" and the interlocutor's "Is so-and-so there?"--time in which the auto-dialer is connecting queued operators. I could probably hang up before I hear a human voice and still have a 95 percent success rate at identifying them.
Weird. I haven't had a college professor who has to typeset mathematical equations who doesn't know LaTeX.
I've used nginx and have been able to adapt old htaccess rules to get everything I used to get with apache. It's good on memory and I've never had a problem.
You mean these?

http://en.wikipedia.org/wiki/Autostereoscopy

You made me google it; I had no idea those existed. At first sight, it looks like it should be able to do what you ask. Geez, it could even diagnose your (short|long)sightedness|astigmatism and recheck it every now and then.

Pretty sweet tech; I hope it gets commercially viable soon!
How much is it costing you?
Out of curiosity, how many times did you apply to YC? If more than once, were any applications with something other than fleaflicker?
is zecter (http://www.zecter.com) a misspelling?
have you thought about migrating your home page to arc?
http only - we're using Apache at the backend.
I'd be surprised if they use the X for anything other than simple hiding.

This was a feature I was pretty interested in about two years ago. I spent some time trying to use the pre-Application-Platform API to build it out myself, but there were too many limitations on the FQL querying API.

At the (relatively small) Facebook developer party at SXSW in 2007 I talked with one of the Facebook employees about wanting this feature, and he said they had it working internally, but that they faced similar problems with the user experience; for instance, they would get all of the people they know well but hate.

I imagine that they held back from releasing it until they came up with the "X" dismissal feature so that users wouldn't have to be continually annoyed by their enemies.
Thanks.. I wouldn't ever try and fake it. All the interest we have is real (we haven't gone after funding yet, all the VCs have contacted us) as I mentioned before, I'd rather not get involved in tactics. Always better to let the product do the talking. But appreciate the fact that I shouldn't explicitly give names.
"It's open source software, which means it's largely unfinished and you can't guarantee that all the math routines are actually doing the right thing."

Um... I think you have a better chance of guaranteeing a routine works when you can inspect what it's actually doing below the hood. Something like MATLAB is based completely on trust (albeit, deservedly).
thanks for the tips. i guess the weird thing about our situation is a lot of VCs have contacted us. we're super early stage and weren't explicitly looking for funding. but as a result, we've had some awesome meetings and built some really good relationships. is it weird for VCs to contact entrepreneurs directly without referrals?
I think what he is saying is that the generation and extraction of concepts requires context that language may or may not provide. If you think about the approach of AI with language, it is almost the opposite of how humans learn: presentation of grammatics/language first and the extraction of concepts from this, whereas humans learn basic concepts of reality before speaking, and language is a partial representation of what they know.
What about correcting for color blindness? There must be a relatively easy way to auto correct certain colors so they stand out more. I bet there'd be a market for that.
I think this means a lot of people have jobs that require a fixed number of hours (say, 40 hours a week and 50 weeks a year) in exchange for a fixed amount of money. Americans may feel the number of hours is too high, but their wage per hour is fine. Basically, Americans work too many hours, and therefore would like to buy hours back.

E.g. Someone makes $100,000 on 5,000 hours per year. They can't work less than 5,000 hours, or they'll get fired. Their wage is $20 per hour. However, working 5,000 hours a year is painful and the dude doesn't need to spend $100,000 a year. He would happily buy back time at $20/hour. If a genie offered him free money or an equivalent amount of free time (with a $20/hour exchange rate), he'd take the free time.
Just add scaling!
http://news.ycombinator.com/item?id=48816
Thanks - I got a basic Nginx config up and running on EC2 within a couple of hours. I'm intrigued by the idea of using it in combination with HAProxy though, so wondered if you didn't mind a few follow-up questions ? Essentially:

- Do you run the two products on the same physical machine ?

- What does Nginx do that HAProxy doesn't ? (i.e. why not just stick with HAProxy ?)

- I noticed that Nginx has some weighting options for load balancing (see http://wiki.codemongers.com/NginxLoadBalanceExample). Are these just not sophisticated enough for your needs ?
What to Do About That Big Project You’ve Been Putting Off
Does KVM allow you to run XP or Solaris ?
ultramonkey in front of apache / php / postgresql

It was very smooth to setup with debian, but my first experience setting it up was with redhat and that was tortuous
Concurrency with Python, Twisted, and Flex
I put in $.30 each and I got two different, correct answers in less than 10 minutes, formatted in an anchor tag as I asked. One was really good, they gave a full proper citation and I had asked for a bastardized citation.

I have a bunch of different requests in for similar tasks now from $.05 to $.15, none of which have been answered in the 45 minutes since I posted them.

Edit: $.3 was way too much, I got an answer in 50 minutes at $.05.
Weren't faxes around in 1989? That's what the mailslot really is.
Ask YC: what happens when the business fails?
I know it isn't guaranteed. However, the odds of failing is greater than succeeding. So what typically happens when the business fails? How do things settle for the founders usually?
Microsoft walks away from Yahoo offer
Microsoft Withdraws Yahoo Bid; Walks Away From Deal
in japan they have a robot to bring you beer.
http://www.ohgizmo.com/2006/01/26/asahis-beer-pouring-robots...

so everything is complete. job accomplished, now if only we had all of the things they predicted in the 80's that would be around by 2020, like robo-cop, and hoverboards.
Neither ActiveRecord nor Hibernate have the words "smart," or "magic," in their names. They were bad examples.

The problem is not with these two packages, but with the developers who use them without taking the time to figure out exactly what they do. Just because something makes a task convenient or easy, does not mean you aren't required to understand how they work.

It's copy/paste, cargo-cult programming, and you can't blame the tools.
Ballmer says clearly in the letter that they will not be pursuing a proxy nomination battle. Do you think he is telling the truth?
"Anyway, I write way too much on here..."

Not at all. We need more war stories here. Thanks.
Agreed. Why is every future always depicted with us suddenly losing all fashion sense and running around in unitards with dorky boots? Its a problem that's plagued sci-fi and futurology since at least the 20's as far as I can tell.

My vision of the future is that we've enabled such accurate control of our environments and such precise manipulation of our genes that we are all naked, all of the time, and its OK.
His explanations seem credible. There are a lot of "poison pills" that Yahoo can swallow between the start of a proxy battle and the installment of MSFT-friendly management.
I'm using Pound right now in one large installation, and nginx in another smaller installation, but it still has decent traffic.

I definitely prefer Nginx. It seems much faster, and has definitely given fewer headaches. It seems like an issue with Pound crops up every few weeks. My Pound config file is about 600 lines long now, and it is starting to get unmaintainable.