skysight-inc/HackerNews huggingface dataset:
Yeah, certainly at the small scale there can be a myriad of reasons why a system doesn't perform. At a large scale its hard to imagine real world scenarios where storage has not been the bottleneck. Regarding whether twitter should be using a database, unless they dont want to store there data they need a "database". Does that mean sql? No. It just means storage. As paul Graham has demonstrated a file system can be a great database. But it still a database.
New TV show covering tech, politics, pop, and arts, powered by Reddit
I think that names shape the way in which you think about your product, as well as the way that other people you talk to respond to your idea. Hence, a good name up-front is important IMHO.
When trying to select a name, think about the message that the name sends, the impression that it leaves, the way it primes whoever hears it for the first time. It doesn't have to be descriptive of what you're doing, but it should probably convey your general approach to how the company should be - e.g. serious vs. fun (personally, I prefer the "fun", but it's really a personal thing).
Why is it shameful? It's not a "wacky correlation" - it's arguing that there is causation. It's basically saying "we have data proving the Laffer curve exists." That's not wacky. That's giving credence to a theory.

Low, flat-taxes are helpful but not politically tenable in the United States.
This isn't about closures. This is about using continuations on the server to store client state.

Surely they could come up with some way to store trivial state in the URL's GET params, without making life too miserable for the programmers.

Nearly 10 years ago, Nielsen wrote about "URL as UI" <http://www.useit.com/alertbox/990321.html>. URLs that break after an indeterminate time are not good UI.
Agree in general, with one note:

> capitalist market responses would be vastly better at "dealing" with it than anything the gubmint could do with carbon tax revenues

Carbon taxes have two sides, the costs and the revenues. The costs do work with and contribute to the capitalist market responses, by making more polluting products more expensive, relative to the less polluting.

On the revenues side, I agree again, but why not use that money for general expenditures? Why not tax the bad things (pollution & other negative externalities) more, and the good things less (sales, income, investment)?
This seems like a fundamental shortcoming of the continuation-based request approach.

Nope. You just add times to the closures.
One thing you don't address: where do messages go while a resource is self-splitting? How much redundancy do you build in to prevent this from being a problem? How much backlog do you expect to handle?

And have you checked latency serving out of EC2? Our experience is that the connections are fairly fast but the latency establishing the connection is significant, especially from the west coast where twitter is based.
First visit thoughts :

First load was extremely slow, later pages loaded at a correct speed.

The menus at the top are very confusing. The first impression I had was "oh gee so many stuff at the top. What a load of crap, I feel like closing the tab immediatly".

At the top, there are 4 differents rows containing lists of links. I'm lost!

At the top of the page there are too many different styles (different sizes, form, color, gaps, ...)

I have not a clue what IPL/ICL means

On the home/My stuff/ My... bar the effect to display the submenu is annoying (slows navigation down).
The fact that on that bar some elements have no submenu and must be clicked on directly while some other have submenu and can't be clicked on directly is disturbing.

What's the difference between "R4E Spy" in the top and the one below?

My stuff, My friends, My page. Why so many "My ..." What's the difference between them?

I MUST click on one of the two option to access the details of a rivalry. Being forced to click on one of the two options when I just want to look at the details of the rivalry without voting is unpleasant. Why can't I click on the vs image?

The name of the two options are in the middle above and below the vs instead of being above each image. It's hard to understand what name correspond to what text.

The pages for each rivalry comparison is overly confusing. There are too many different styles. It's not obvious what is a link and what is not. The separations between the different parts of the page is not very clear making the whole thing hard to read.

Energy Meter bars are too small. (What the hell is it anyway).

"Top 3 Fans of ..." Why are those three the top fans??? Why them and not other?

In the Featured Rivalries box (and other places), It looks like the circled 1 is crossed. That's odd.

On the registration page the password and confirm password fields have extra spaced. If there's two fields that should NOT be kept separated in a form it's those two!

I have to "Join this Rivalry first!" before being able to add a comment on the Passion Talk? What a hassle.

When I click on the Join this Rivalry link I'm harassed by even more crap! No I don't want to get updates, I just want to leave a comment! ( If I want to get updates on that rivalry I'll opt-in by clicking on the relevent link on the rivalry page).

Wait it's not over yet!! Why ain't I brought back on the rivalry main page? Now I have to click on the "XXX" highlighted in red in Congratulations !! You have joined ''XXX''. Damn! It's not a link... Why is it in red then? (By the way only one of the two apostrophe is in red)

Now what? There are three choices but the icons are all but explicit about what they do (especially the Add talk one). I have to read the text to understand.

How I am supposed to edit/add my picture?
It's a trade off for running the entire app in memory.
If you look at the URLs for voting on comments, they aren't using continuations. I don't see any reason why comments should.
Using that logic though, why have it on comments but not on posts?
Speaking from experience: I coded many thousands lines of Ruby code for my startup all alone. Then we raised seed, and hired a few additional developers. Two of them joined my project and took over the development from me. They're both very experienced programmers, but had no prior Ruby experience. It took them less than a month to become productive. Srsly.
In short, if you write good code and bring in good developers, Ruby probably isn't going to be a problem.
I don't understand why Marvin Minsky hates America so much that he wants to put the government in charge of assessing such risks. The government is an incredibly complex device for distilling a tiny amount of individual ignorance from each voter into a moonshine of batshittery.
rpm system was not enough, but yum changed the state of things
> The Perl community was too unfocused on the web.

That's a fairly ridiculous claim. Perl CGI had all the momentum. It had plenty of forums. I repeat: PHP won because as the internet grew and compile-at-run CGI was no longer sufficiently scalable mod_php was more viable than mod_perl or fastcgi.
Poetic license, yes.

And I have felt the same way, seeing "self" everywhere.
You are right, I meant continuations, sorry.

HN is using URLs for passing those hash keys (if they are hash keys of course), but they need to store them on the server as well. So URLs themselves don't solve the problem in this case.

It's actually that HN doesn't use databases, where they could keep session-specific data much longer.
If you equate transhuman AI with godlike powers, then it's hard to argue against it escaping. Then again, I bet I could train a dog to keep most humans trapped in a cave, provided I chain them at the opening.

What if the researchers communicating with the AI had no other access to it? What if the physical plant and the administration of the computer systems were off limits to the researchers, and they had no power to release the AI? Furthermore, what if the AI were not allowed to know anything about the researchers? The researchers could be forbidden from revealing anything about themselves to the AI. This would make it impossible for the researchers to publish anything, but let's say that they are working for an organization like the NSA.

It would be really hard for the AI to break out. But it would make for a good science fiction book!
It's NumOfMessages, not NumOfUsers, and Messages = Messages/User * Users. Then the question is whether Messages/User and Recipients/Message remain constant as NumOfUsers goes up. If both do, then your total cost is linear with the number of users, which is what you want. But if either of them is linear in the number of users, then the total number of messages becomes quadratic in the number of users: a linear factor from growth in userbase, and a linear factor factor from increased usage by each user. Most likely there's a logarithmic correlation (Recipients/Message goes up as more people are on the service, but not by as much as total users), which is still generally fine for scalability concerns.

E-mail's actually a really great example. It worked great as long as it was used for person-to-person communication, which held both the number of messages/user and the number of recipients/message constant. Then spammers found out that the more people who had e-mail, the more they could send to at once, and sent recipients/message through the roof. In 1997/98, there were actually a number of articles about how the e-mail architecture was in serious trouble, and unless some way could be found to limit spam, it would collapse within a couple years. Of course, they dealt with it through spam filters and blacklists on the server, so now e-mail works fine.

Twitter's problem is that both Messages/User and Recipients/Message average significantly higher than e-mail, and yet the resources they have available are significantly lower than the full resources of the Internet. So even if they don't have an architectural scalability problem, they could very well have an economics problem.
Scribding PDFs here is the new page framing. Imagine if every link you submitted appears in a frame with some ads at the top? That would be annoying (and so 1996), I don't see the Scribd thing any differently here. I have _no_ problem downloading PDFs.
YourWeek: a reddit-powered TV news program by PBS
James Simons: former mathematician and now billionaire hedge fund manager
Kevin Burton of Tailrank released his own thumbnailing code earlier this year at http://feedblog.org/2008/05/10/tailranks-open-source-web-thu... .
how good is SliceHost?
OK, thanks for the link
that's against the rules :)
When they've got 3 others who are more stable! :/
> When an engineer gets a new idea and develops a new algorithm, we test their ideas thoroughly. We have a team of statisticians who look at all the data and determine the value of the new idea. We meet weekly (sometimes twice a week) to go over those new ideas and approve new launches.

This line has to make a lot of search startups more comfortable. In one sense it's scary that Google has so much manpower behind keeping their results high quality. But it's quite clear that they cannot adjust on a dime and try wild new ideas. This is where the agility of search startups can try out different approaches that might not "maintain quality" against legacy search results. It makes them a bit more iterative and hopefully some startups will make some interesting leap to get even better results.
"But the fact is that attacks by strangers are so rare as to be practically nonexistent. If your child is assaulted, the perpetrator is almost certainly a relative."

Maybe that statement is true in today's world. But for tens of thousands of years, while our brains evolved, I would guess attacks from strangers were a lot more common.
Why not? It's a one-in-four (or one-in-three, depending on who you believe) chance of being rich in a few years--that means that in the worst case scenario, if you keep at it, starting over when you fail, in about 10-15 years you're very nearly guaranteed to be rich. And you almost certainly won't starve in the process.

Of course, that assumes a reasonable level of intelligence, education, and drive.

Startups are about the best game going, as far as I can tell--I wouldn't be playing if the game was rigged against me (more than a little, anyway...sure, small companies have higher relative regulatory burden, but on the whole the technology game is actually rigged in favor of new companies, from a growth perspective).
I'm not exactly sure what a "moonshine of batshittery" is, but you've got my vote.
Another profile:

http://www.bloomberg.com/apps/news?pid=20601109&refer=ho...

For folks interested in understanding his strategies, here is a thread by some quants:

http://www.nuclearphynance.com/Show%20Post.aspx?PostIDKey=48...
I don't understand your correction, but my lack of understanding is really immaterial. I actually agree that twitter has a problem with economics if they cant sell anything. Whether it is worth solving the scalability problem is a good question. Certainly if they keep growing with no revenue, at some point it is definitely an economic problem. Indeed I posted a while ago about the community taking over twitter as a distributed public service (like email). I do think that there is an upper limit to the amount of messages the average user will send.
His fund suffered loses recently, making it all suspicious that the math was just good old "sweeping the risk under the rug" kind:

You buy a decent assets that give you 7% return, but have a 2% chance of default. You borrow at 5% and leverage to the max. As long as your assets perform, you have returns that beat the market consistently with high probability. But with a low probability you lose a lot more, and you just hope that this doesn't happen while you rake all the fees and gains.
And when a large proportion of families (broken, dysfunctional, otherwise unable) fail to teach kids what's important: what then?
i declare you the winner.
> they don't quite understand wanting to turn down a 100k job to bootstrap something with a very small chance of success.

Your average income will exceed US$100,000. You have a small chance of becoming a billionaire and a larger chance of getting nothing. You can repeat this process five or 10 times before you succeed - or take a US$100,000 job.

The contention is that parents see your likely income being below your true potential.
"Wreck a roast chicken and it’s $12 down the drain. Wreck a product launch and there goes your career..."

God help us if that was actually true. Temporarily hurt it, maybe.
I LOVE Arch Linux. It is blazingly fast, even on old computers. My favorite part about it is it never does anything without you telling it to. You set up everything when you install it. You configure the installation, you pick the Desktop or WM. You set up the networking.

They also have a really good package manager and AWESOME user repositories that have pretty much anything you would ever need.

Honestly, I got into Linux because I wanted control over my computer. I started out with Ubuntu because everyone says it's a great place to start. I got fed up twiddling with GUI settings and undoing what Ubuntu had done. I installed Arch. It took a couple of hours, but at the end, I had a computer that I set up entirely. I knew everything I did and how to change anything should the need arise.

Also, if you have an easily configurable internet connection, you can download the FTP install ISO which is all of 30MB and get up to date packages right from the start.

Their user community is also fantastic. Check out www.archlinux.org for more info.
I addressed this in another answer here but happy to do it again. The way shard splitting works is that the initial shard keeps working while the new shard is being built. During this time, all messages to the old shard are written to the old shard and the new shard, so, unless I don't understand your question I dont see there being a backlog.

Regarding EC2, latency isnt really an issue because Twitter is far from real-time. That said, haven't really noticed any problems with EC2 connection latency, but we are not live yet so I would defer to you on that.
Giles Bowkett: Summon Monsters? Open The Door? Heal? Or Die?
I must admit I totally didn't get it. What is the purpose of that algorithm? And why the dow-number, looks like snake oil to me?
Yes, I didn't intend for my comment to be news to anybody who uses Lisp. I just wanted to point out why list processing is still important.
Makes no sense to me. Never did. The speed which the code can be understood says much more about the writer than the reader. I have business analysts, project managers, and users that can read and understand my code. Do you really think that's a valid test of "programming ability"?

Also, consider:

1. 30 pages of well written code != 30 pages of poorly written code

2. 30 pages of well documented code != 30 pages of poorly documented code

3. 30 pages of high level code != 30 pages of low level code

4. 30 pages of COBOL != 30 pages of BASIC != 30 pages or Java != 30 pages of Python

5, 30 pages of Microsoft code = 1 page of mortal code

The finest test of programming ability is what you can build that satisfies requirements and works perfectly. I'm not sure Bill Gates ever understood that.
Why You Should Use LinkedIn
Why You Should Expand Your LinkedIn Network
How did this comment end up in the rss feed?

Interestingly, the "comments" link for it points to http://news.ycombinator.com/item?id=196342 -- the original post without any comments shown!
Totally. As a total newbie, I picked up a Perl book and got frustrated. Then picked up a PHP/MySQL book and built a few websites.

The biggest thing for me, at the time, was the ease of use of PHP/MySQL. I haven't touched Perl in nearly a decade, perhaps it got better, but even in 2000-2001 PHP made it very easy for a total newbie to build dynamic websites.
The whys:

1) Capital and advice for fast growth

2) Control, flexibility, or you're going after a market where growth is necessarily slow.
Unless I'm missing something, M-expressions as expressed in the Wikipedia article are an alternate syntax for S-expressions; all the examples therein have the same number of parentheses/brackets in either syntax, so I don't think those M-expressions will save you many parentheses.

That said, you really only need parentheses (or other grouping operators) in situations where you don't know how many arguments there are; in most function calls, for instance, you know exactly how many parameters are needed, so the parentheses can be eliminated. This is effectively how Logo and Forth work.

However, eliminating parentheses in this manner means that the syntax tree interpretation of any given code depends on the function definitions currently in memory -- which is not the case with Lisp code, whose structure is explicit. I assume this would complicate a macro system.
People didn't worry about that (i.e. formal education systems) until only recently, but I guess what's "important" has gotten more complex and the rate of "dysfunction" has increased.
Interesting. Sorry to hijack this post but I want to know:

1) Which is better: Ford or Chevy?

2) Should I use Linux or WIndows

And don't forget the most important thing: Why?
That sucked. Twice.
The reply link I used to add this comment, had timed out, which seems wrong.

Have continuations just moved the the complexity -- making some aspects more elegant, while resulting in undesired side effects like expiring reply links?

I find that often happens when I'm designing abstractions - complexity moves somewhere else, and I later get unexpectedly bitten.
I don't know many good answers to that question - but in my bones I know that the federal government isn't one of them.
> the main reason we still PHP used so much, is that a lot of sites were build before 2005

That's really it - look at the dates on the sites he mentions. Wikipedia - 2002. Digg - 2004. Wordpress - 2003. FaceBook - 2004. Flickr - 2004.

The big hits of 2005 were YouTube (Python), Reddit (Python), and Twitter (Rails). Then there haven't really been many big hits since then - the ones I can think of with the most publicity are Pownce (Django), FriendFeed (Python), Justin.TV (Python), maybe Kongregate (Rails). So around 2005 there was a big shift away from PHP and towards Python/Ruby solutions.

There was a similar shift around 2000-2002 from Perl to PHP - the big hits then were LiveJournal (1998, Perl), Del.icio.us (2002, Perl), Blogger (1999, dunno), HotOrNot (2000, dunno), Craigslist (1996, Perl), EBay (1996, Perl), and IMDB (1998 or so, Perl).
I presume you posted this in an attempt to piss him off? :)
My startup is not making this mistake, but does anyone care? People seem to prefer trolls and fun social noise to high signal streams that demand your attention.
http://wiki.xkcd.com/geohashing/Main_Page
Hank's proposed solution is expensive; we should just admit that up front. But is it more expensive than being down and fighting fires all the time?
Here's a good overview from one of the Symbolics folks, DLW (Dan Weinreb): http://dlweinreb.wordpress.com/category/symbolics/ .
The Dow provides an unpredictable (if you can predict it, you probably have other things to deal with) random seed that determines a location inside of a consistent geographic region. For example, here is today's location in the East Bay area:
http://irc.peeron.com/xkcd/map/map.html?date=2008-05-21&...

This allows for unpredictable, spontaneous meet ups of like minded folks.
Medallion was up something like 70% after fees last year. Are you talking about REIF, which is benchmarked to the S&P, and which performed close to the S&P?

Edit: Now is as good a time as any to disclose that I work with Rentec, among other companies. I'm not sure if this makes me informed or just biased, though. Everything I've said about them is publicly available.
I believe its purpose is to generate a random different location near you every day... and then you are supposed to go to that location sometime during the day and meetup with all the other people that are following the algorithm
If you work at Microsoft (or somewhere else with teams and long-lived products), you have to read a lot of other people's code, so its true.

If not, maybe not so much.
Also, Symbolics, LMI and TI all faced the big problem that their machines were in the $100K range, which was too much except for well-funded ARPA projects or big-company research groups. (E.g., I was one of the original system hackers at the Schlumberger/Fairchild AI Lab in 1980, and, with Schlumberger's deep pockets, we bought several Symbolics machines.)

The newly-appearing general-purpose workstations (more like $10-15K) from Sun and Apollo ate their lunch, since you "can't bet against the desktop." I.e., no one ultimately wants to share hardware (which is also what killed the DEC VAX and special-purpose minisupercomputers like the Multiflow VLIW machines we built in the early 80's).
The Class Everyone Thought You Took But You Didn't
Why is friendfeed on this list?

And does anyone else consider tumblr closer to friendfeed than twitter/pownce/jaiku?
100 Attributes of Successful Entrepreneurs
It's my site, actually. Thanks kirubakaran for posting the link:-)

Frustratingly, that's about as good as I seem to be able to do in terms of design. Another good reason to get a cofounder who covers for some of your skill gaps, I guess.

In any case, yeah, I'd love for more people to contribute, at least to discussions, if not to the summaries. As note takers like ajju below have found, I absorbed the material a lot better by creating some of the summaries, so there is some benefit for summary creators, beyond just the value of sharing.
Where are the IMs (MSN, AIM, Google Talk, et al.), email, and IRC?
It's not just the probability of an occurence that leads us to behave in an unreasonable manner. The other important thing is the impact of that 'rare occurence'. Most critics don't seem to take that in to account at all...
I'm talking about Renaissance. Medallion is closed and manages money only for Renaissance employees.
O'Reilly AlphaTech Ventures Startup Camp
Gallery of Network Operations Centers