skysight-inc/HackerNews huggingface dataset:
Hm, what about FedEx-ing a HD with the encrypted data? It might be faster/cheaper/more practical?

Edit: That's assuming cperciva has physical access to his server, rather than working with e.g., EC2, or that he has more bandwith to spare.
I love it. Though I'm not an expert in optimized configuration of servers. I guess that's sorta the point.
Alas, my lawyer explained it to me like this: There is a front door (that allows people to submit paperwork), leading to a back door (that gives people Green Cards). You're already through the front door. All they're proposing is that the front door should be bigger, but they (the USCIS) don't have the resources to process even the people who are already through the front door and push them out the back door. Nobody seems to be proposing more resources so that the USCIS can get people through the system more quickly.
Kids these days, with their hair and their twitters...
Well, server optimization has been my primary hobby for the past month, and I'd like to make some clarifications about a few things the article says.

Saying that Lighttpd supports "CGI scripts, Ruby on Rails, Pylons, TurboGears, Django, Quixote, webpy, Mongrel, Catalyst, Mason, Joomla, AWStats, and so much more" is kind of misleading. Lighttpd can do a few things: it does fast cgi (fcgi support explains about 75% of the examples in that list), it can serve static files, and it can proxy requests to other servers. Lighttpd doesn't "support Mongrel", because Mongrel is another server. What Lighttpd does is use something akin to mod_proxy to send certain requests somewhere else, that somewhere else might be a Mongrel cluster that is running on port 8080 (hence it "supports" Mongrel).

That proxying service is where the majority of Lighttpd usage has been. A typical efficient setup these days (YouTube does along these lines) is to have lighttpd accept incoming requests and if the request is for a static file then it serves the request (because it is very lightweight (i.e. minimal) it is more efficient than Apache at serving static media), and if it is for dynamic content the request is passed to Apache, which then passes it back to Lighttpd, which passes it back to the requester (this means that "heavy" Apache threads will be dedicated to a request for a shorter period of time since it only has to communicate/wait for Lighttpd instead of the actual requester).

Lighttpd on its own isn't necessarily the best solution (an example from my experience is that mod_python and Apache become more efficient at serving dynamic content than fcgi and Lighttpd as the numbers of requester per second get very high). Usually a combination of a light frontend server and Apache in the backend can provide the best performance (again this is at high numbers, at lower numbers lighttpd is probablly equally or more efficient).

As a final note, I'd really recommend Nginx over Lighttpd if you are looking for a light http server to serve static content, support fcgi, and/or proxy to other servers. The biggest reason? Lighttpd leaks memory and has to be restarted occasionally because every couple of days (under heavy usage) it dies. Not a great feature in a production server.
Allow me to introduce you to my corollary to Greenspun's Tenth Rule: "Any sufficiently complicated Scheme or Lisp program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of object orientation."
Valleywag on Facebook's Parakey acquisition
His argument that you have to pay market rate regardless of the equity equation is false. If a transaction is a net loss for the company, you shouldn't do it.

If your situation is so desperate that you feel forced to hire someone (or take more funding) at what seems an excessively high price, then what's really going on is that you're overvaluing the company. If your situation is desperate, your value is low.
What if Facebook loses my data?
Facebook has never, as far as I know, claimed to want to be a computing platform. There are lots of senses of the word platform, but sense in which the current FB is a platform is not that one.
I must never have written a sufficiently complicated Lisp program.
Plaxo With Sauce: PIM Site Now Supporting OpenID And Microformats
Plaxo is the latest high profile web service to announce support for OpenID, the standard for online identity management.
By all analyst accounts, Facebook is not the next Google. They have no "real" revenue other than fixed revenue that MS has to give them each year.
Also well said. And it's worse than that - when you allow your users to log in with OpenID, you don't just give up the marketing value of their email - you give it away to a random third party somewhere on the Internet!

Your customer's OpenID provider gets to know her email, her personal information, the sites she visits, the exact times and dates of every login to those sites, even her passwords. The provider also gets to show the user a page at a critical time - the time just before the user signs on to a new site. (Or is it every time the user logs in? God, what a bonanza that would be.) This is the kind of information and access that Google has turned into billions of dollars - only it's potentially more detailed, and more complete. It sounds like a DoubleClick executive's dream come true. No wonder that every big Web company is falling over itself to become an OpenID provider, even though very few sites actually support OpenID logins.

What's the average business willing to pay for an ad on its competitors' actual login pages? We may soon find out.

So, yeah - without even getting into the potential security issues, the very real user-confusion issues, and the terrifying ten-percent-of-my-users-can't-login-because-of-a-bug-on-someone-else's-site, now-wtf-do-I-do? issues, I'm not sure how many sites outside the Geekosphere are ever going to support OpenID.
Ok, looks like he was.
A few applications that built this functionality have been taken off-line by request for TOS violations

So that's what happened to File > Save As in Internet Explorer.
with 300 friends, keeping up to date contact information with "Save As" isn't really going to work.
I can understand why they think it's a good idea to ban such applications, but it's not going to actually accomplish anything.

If I was a malicious marketer that wanted to harvest cell phone numbers, it just means I'm going to have to save them by hand.
Ever see the movie "Eraser?"
There used to be an option to export to CSV. Looks like they got rid of it. The fact that they are actually removing functionality says something.

A couple years ago I exported all my Facebook contacts into a mind map using FreeMind. I then sorted all of my friends according to where I met them. So for example, a possible node location would be College -> Classes -> CS100 -> John. Then if a person had introduced me to more people, those new people would fall under that person. Not only was it useful as a visual rolodex, but it was also useful for visualizing which people and organizations were the most socially productive for me.
That would be a great feature for Facebook to implement. I wonder if it's possible to get around the TOS if the application were to check with each user before sending their contact information to the person who requested it.
"All they need now is a good way of monetising.."

Quote of the year.
seeing as they already agreed for that information to be available when they added the user as a friend, why would you make them agree again?
Locking up your users' data ensures that there is an increased cost associated with migrating to a new social network.

They're just protecting their share of the market.
This type of 'walled garden' strategy has been disastrous for so many companies. Hope they change their mind.
Gov & Bush admin. has allowed 30+ million of unskilled, illegal aliens to enter and is fighting for their amnesty and yet they're making it extremely hard for highly skilled people with US college degrees to stay and contribute to job creation. This is an insane policy that needs to change.
How to operate before incorporation?
What is the best way to run your startup before incorporation? For example if I were getting a startup ready for Y combinator and you recommend not incorporating beforehand, how do you recommend handling any business transactions that would occur in the meantime?
Suppose your present hope of future utility from your startup varies
linearly with the number of shares. (This is not a radical assumption;
essentially all startup shareholders feel this, at least for numbers
near that of shares they own.) Suppose you trade 6% of your stock in a deal that will increase your average future utility by 6.4%. You've made a straight trade of hope of future utility for future utility. We don't even have to introduce money.
Actually, if someone is wrong about math you probably should tell him so.
I guess I beg to differ. I can think of many examples where providing proprietary services for locked-in users has been a primary source of competitive advantage -- consider why Microsoft fights so fiercely to maintain a high market share for a product which generates no direct income (IE). It's a little counter intuitive at first, but they're ensuring control of the market. That's a powerful thing.

Referring to facebook as a "Walled garden" brings to mind AOL's demise, but just because they may appear to be similar tactics the companies aren't doomed to suffer the same fate. AOL failed for a number of reasons, the largest of which was a failure to adapt to the needs of the market. Facebook appears to be a more nimble company and I would be surprised if it had similar problems.
For me, instance messaging has replaced the telephone. It is, frankly, better in every way when discussing technical matters.

Email is better when you need to make a more elaborate statement that doesn't require back-and-forth communications.

I have had business email written in the style of text messages. To me, this is offensive, and it's a trend I hope ends.
If you're running Rails using Mongrel, beware that Zed Shaw (Mongrel's principal author) has stopped recommending lighttpd: http://mongrel.rubyforge.org/docs/lighttpd.html
How to fix Copyright, RMS' views
Copyright developed in the age of the printing press, and was designed to fit with the system of centralized copying imposed by the printing press. But the copyright system does not fit well with computer networks, and only draconian punishments can enforce it.
yc may see application surge.
While I don't agree with all of RMS' view points, it is still a very interesting talk. Also they managed to get the Q&A boosted so you can actually here the questions! :)
try cofoundr.com
What problems does Marketing solve?
Very popular BS these days. facebooks, myspaces, linkedins and friendsters come and go, but nobody has suggested anything even remotely close to power of email.

Spam, along with viruses, spyware and adware (IMHO) is mostly a problem for computer illiterate: those users will be gradually dying off.
Sounds very useful. You know what would be more useful than CSV? A FOAF (Friend of a Friend) profile.
Good news! Civ 4 works on OS X :D
How does PayPal calculate fees for international and cross-currency transactions?
PayPal has about 12 different fee help pages for the different countries, and reading them makes me confused. Specifically about international and cross-currency payments.

If I'm a seller with a US-based PayPal account and I want everything in USD on my end, the base rate is 2.9% + $0.30 USD. If the payment is coming from outside the US, the fee is 3.9% + $0.30 USD.

And it looks like there's an extra 2.5% put on top when there's currency conversion involved. But when is that 2.5% added? Another related thing I don't get: PPCalc says that "fees are calculated in the sender's currency", see http://www.ppcalc.com/faq.html#faqlink05

Do I have to know what country the user's in and what currency he pays with to determine the fee? In other words, will I only know the exact fee afterwards when PayPal tells me what it is? Problem with that is ... we want to show our users something like "To top up your account by $100.00, you'll be paying a total of $103.30 including PayPal fees. Go ahead?"

Anyone know how this all works, or know a nice, simple page which explains it?

(Yes, I know about PayPal Sandbox, but the sandbox user profiles have limited options, and I'm not sure I'm getting all the country/currency options.)
I like the idea but I dot believe it will work. Major players (Amazon, Google, Yahoo, Microsoft) are NOT interested in open standards for their user authentication. This is why Microsoft Passport failed and this is why Google Checkout will fail and OpenID will follow their "lead".
I agree. In private.
The "rise" of lighttpd? I think it's dying now. Apache 2.2 and Nginx are the kings of the hill.
Thanks for the well wishes, guys! Today was our first day at Facebook and we're excited about all the possibilities (which I'll blog about soon :)

I regret that YCombinator isn't getting more exposure out of the acquisition, since they were a key backer of Parakey.
You should tell him he's wrong, but you shouldn't be a jerk or gloat about it, which cperciva wasn't.
VC Model, How to Fix it?
What about the other half of his argument, that you shouldn't pay more than market rate?
Perfect Timing Doesn't Exist. Stop Waiting For It
The only interesting part was that PG, a quintessential rational man, is apparently a vegetarian. My brain exploded.
Of course, anyone could write a Python script (or even lash together some shell script involving wget) that would spider your friends' Profile pages and any of your own relevant information (all messages and wall-to-wall threads). Then you would have your own local copy of everything, and would no longer be at their mercy if they suddenly closed up shop.

Hell, you could set it up as a cron job that's done every night. It would look to Facebook almost like the kind of activity I've seen from several users, clicking through all their friends to look at pictures, wall posts, and other crap.

I'm not saying that the current setup is optimal; I'm just saying that backups are certainly not impossible.
Yeah. I wish i could have my time back, but nice rail gun.
The big win of lighttpd/nginx is that they're single-process [1] multiplexing servers. Apache spawns dozens (or hundreds) of processes/threads and relies on the OS to distribute resources. This critical difference is why they're so much more efficient and generally faster. The epoll mechanism in Linux has widened the gap even further.

Brad Fitzpatrick created a software load balancer in Perl that handles all the requests for LiveJournal based on this principle.[2]

1. A small number of processes may be used to take advantage of multi-processor machines.

2. http://danga.com/perlbal/
For most people the big improvement is from just two things are 1. Use a French press. 2. Grind your own fresh beans. That puts you ahead of 99%+ of the coffee most people drink everyday.
I don't do the daily grind; I don't even by beans.

But I do use a french press, and it really does make a big difference.
No, really.. It is wise to never publically tell someone they are wrong, unless you're defending someone. You should wait until his friends and colleagues have left his side, then whisper your opinion into his ear. In the Internet world, that means sending him an email. To not do so is bad karma in every sense of the word.. My post above was modded down because I publically pointed out that he was wrong to point out Paul's percieved flaws in public; maybe I should have sent him an email instead. This wisdom is proven true over and over. My #lisp fiasco convinced me of that (long story).
It seems every 6 months or so, one of these single process, select based servers comes out as the "apache killer", only to fall prey to a later successor. Anyone else remember boa?
I'm surprised this discussion is even happening. This is 2007, and Macs can run Boot Camp, VMWare, Parallels, CrossOver, VirtualBox, CrossOver and Cider.
we moved from lighttpd to mongrel a few months and couldn't be happier.
That's not really true at all. BOA did almost nothing but serve static files and there was no consistent development. Same is true of thttpd and all the rest.

Lighty and Nginx are becoming full-featured multiplexing servers. It's a lot of work to add features when all the code must be super reliable and non-blocking. No serious open source effort has ever been made. Zeus has been the only option for years, but it's commercial and expensive. They've made millions filling this void.
There's reason to believe intelligence and vegetarianism are at least correlated. It certainly matches with my experience. The two smartest I personally know find eating meat disgusting.

http://news.bbc.co.uk/2/hi/health/6180753.stm
I used to use an app called facebook sync. It synced my facebook account with my mac address book and phone. I had all my friends phone numbers kept up to date with no hassle. The application got banned last week and now I'm annoyed at facebook. As much as I like facebook I want to take that data with me sometimes, not even to go join a rival network!
"AOL failed for a number of reasons, the largest of which was a failure to adapt to the needs of the market. Facebook appears to be a more nimble company and I would be surprised if it had similar problems."

You reckon AOL became the largest ISP by "failure to adapt"? No...the failure to adapt came much later. They ruled for a long time. Facebook is barely a wee lad in comparison. If it gets a run as top dog for as long as AOL then Zuckerberg should count his lucky stars.
DBA or LLC. Costs $20 or $200, respectively, and each takes about an afternoon to fill out the forms and file. Or just do business as an individual. If you don't need a business bank account, you can do that too. PayPal and Google checkout will accept money on your behalf without much trouble.
Right, but they're secure enough for most people. Think about it, the chances that someone gets access to my own computer (stealing it, breaking into my home, or hacking it over the Internet) are higher than the chances of that happening to a professional service provider. At least that's the case for most computer users. I heard recently that 25% of all computers in the world are zombified. So much for security on your own computer.

But anyway, that's why I said if you can improve on their offering then go for it. Given the current options, they're the best that I know of; and I say that knowing that they have a lot to improve. So, if you build a better service, I'd consider switching.
You're wrong to tell people they're wrong to publicly tell people they're wrong.

Most mature and intelligent people love it when someone is able to offer useful critiques of their work, as long as they're civil about it. For example: http://en.wikipedia.org/wiki/Socratic_method
http://valleywag.com/tech/separated-at-birth/oomas-arrested-...
Interesting project, unfortunately I didn't found any way to enter in contact with you. You can contact me through my project web site http://www.disnetwork.info

I'll be glad to discuss this remote secure backup project with you. It was once on top of my list for a DIS application, but I dropped it for various reasons that I would be glad to share with you.
I admit I have not thought through the political and economical consequences very far, but it seems like skilled workers are a national resource to be treasured. If skilled engineers want to come, I say let them. Better than competing against them.
Online Desktop - future of GNOME
Havoc Pennington says in his GUADEC keynote that the future of GNOME is the Online Desktop
"... Locking up your users' data ensures that there is an increased cost associated with migrating to a new social network. ..."

means your living in a roach motel. great talk by Simon Phipps, 'The Zen of Free', 18m, 8.3Mb, 2005OCT18 ~ http://www.itconversations.com/shows/detail866.html explaining why restricting communities is poisonous. No roach motels for me.
It makes sense until Gears of War comes out.
Loopt Raises Series B Financing
Video Podcast CalacanisCast #30 With Angel Investor Ron Conway
His address is at the end of the blog entry in which he introduces his idea (the first of the links above).
Is that kind of thing safe, though? With all probability, their robots.txt and user agreement disallows it. So if they detect your spider, they might shut down your account?

Not a good solution...
"Having gone to college in the Facebook age, would I even know who my college friends were, and how to get in touch with them? (Honestly, probably not)"

Maybe it doesn't matter much then...

Still, it's stupid that one can't export data from Facebook. I still wish somebody could explain the Facebook craze to me. Any introductory articles available?
Transmission Control Protocol (TCP) - a classic case for disruption
basically, the idea behind that comment is that while I have 10-15 very close friends that I still keep in touch with on a week-by-week basis, there are 290 other friends who I'd love to see, but would probably not think about out of the blue on any given day.

it helps me keep track of more people that i want to keep track of, and i can easily see a snapshot of their life to catch up on what they've been doing.

while in college, it helped me connect with more people, get the easy questions out of the way (is she single or not?), and connect with people through mutual friends (hey, you know her too?) broken down simply, facebook helped me make more and maintain better relationships more efficiently.

there's nothing nerdy about that at all. at Penn State, some of the earliest adopters were the greek scene.
I know you are well intentioned, but if I'm full of it I'd rather someone told me, preferrably in a respectful but blunt way. I'll form my own opinion anyway.

I know I took criticism personally and reacted very badly once or twice in the past, so I see your point. But I realised I was being a baby and grew from the experience. No speech on humility can make you humble. At best, it will convince you you should be humble, and maybe by acting humble some of it will sink in and stick. Real humility comes from realising your mistakes. [Edit:] That's painful at first, but necessary to get over your ego.

[PS: Sorry, I drifted into replying to other comments of yours, and the end result may be confusing.]
PR 4/10 in 5 months
Not to split hairs, but at different points you said

- that for essentially all startup shareholders, the present hope of future utility varies linearly with the number of shares, and

- that the utility function for most founders is a step function.

Both sound right to me separately, but don't they contradict each other? Maybe most founders just live with this paradox without realising?
By now I am thinking the main feature of Facebook is to give you an aggregated feed of your friends blog postings? That makes sense, although people could just get blogs elsewhere.

I understand the benefits of managing friendships, but still it sounded odd if you say you wouldn't even remember some "friends" without Facebook.
A Zero-Cash Case Study | Bootstrapping the startup
it's not that i wouldn't remember them, just that i can't keep up to date on them easily... i might spend 5 years of my life without thinking of a particular person. with facebook, i can browse through most people i met in college (or later, for that matter) and stay connected.

and it's very far from just getting a friends' blog feed. you can, in 1 stop, check out their recent pictures, view what they've been up to, who's been talking to them/who they're talking to, their life status (relationship, employment), and any and all other information they choose to make available. i can also very quickly glance at someone i don't know, and get a quick feel for what they do/like/what kind of person they are.

a very simple but powerful example: i see all of my friends' upcoming birthdays when i log-in. sending some birthday wishes + a note is a great way to reconnect with people you don't talk to often.