skysight-inc/HackerNews huggingface dataset:
Unless you know what you need to scale to, you can't even begin to talk about scalability. How many users do you want your system to handle? A thousand? Hundred thousand? Ten million? Here's a hint: the system you design to handle a quarter million users is going to be different from the system you design to handle ten million users.

http://teddziuba.com/2008/04/im-going-to-scale-my-foot-up-y....
I really don't see that "dollars per byte" is a sensible way to measure the price of transferring data.

If it makes you feel any better, think of the SMS fee it as a five pence fee for establishing the connection, and then a negligible per-byte cost which gets rounded down to zero cents. Happy now? Is five pence really all that unreasonable a price for sending a message?

Don't get me wrong: phone companies do all sorts of things which are annoying, overpriced and/or unethical, but I don't see this as one of them.
Correct. Blaine Cook blogged (in the post this article riffs from) that they don't use ActiveRecord, for example.

I believe someone has made the defensible claim that most of their activity occurs via clients (twhirl, alert thingy) and SMS over the API rather than through the website. Unless they are using ActionController and ActionView to do the rendering and output there, it seems they would be using "traditional RoR" in a very limited capacity.

I'm not being an apologist, but I think if people are discussing Twitter and scaling for the purposes of learning something useful and not just framework bashing, language bashing or avoiding doing homework, we should probably avoiding talking as if they use the full stack in a significant way as it's a red herring.
Ah, the cycle of incompetence. Mphasis employs too many stupid people to live. Mphasis is acquired by EDS. EDS employs too many stupid people to live. EDS is acquired by HP. HP employs too many stupid people to live, who will bail them out? Taking all bets, folks.
faq it.
Here's a suggestion: compare it to using 10x as many programmers to write something in J2EE because it's easier to hire 10 J2EE drones than one RoR whiz (where "whiz" is understood to mean "person who has experience solving RoR problems").

It's all a question of what you believe to be the scarce resource. H=e obviously believes that for a well-founded company, server CPU cycles are not a scarce resource. This implies that he believes that RoR addresses some issue raised by something else being the scarce resource.
The GNU GPL is contagious. So what?
Ask YC: Have you built a good website that nobody visits?
Did you ever launch a project that deserved popularity, but inexplicably failed? Something you still think could have changed the world? Post a link here.
I think it's because so many sites rely on very simplistic log-in systems for the sake of simplicity. Sometimes it's easier to just make a basic thing on your own than it is to use somebody else's system.
By volume, 20oz bottled water is more expensive than gasoline. And will continue to be so until gas hits $7/gallon.

Smart industries charge what the market will bear.
"an" is used before words starting with a vowel sound. Not words starting with a vowel character. This is a common mistake.

Since FAQ is an acronym, the "F" here does not sound like your average "F". Phonetically it could be represented as "EFF".

While "an FAQ" looks strange, it is grammatically accurate.
The application interface problem
If this is true, what's next?
He mentions that "yesterdays dynamic languages had great performance and tools" - well, I never experienced any of the xerox greatness myself (lisp and smalltalk machines), but in the 90s I was a user of many small talk apps (before I was a developer) - and they were a horrible, horrible experience. I think the people building them got off on the cool development environment, forgot that there were end users taht didn't care how they worked underneath. So I can't agree with that. I think its rose coloured history. A lot of those tools died cause they didn't yield anything useful for end users.
thanks for your viewpoint
Here's something that nobody talks about: people writing articles from different points of view.

We don't get this often. We dont' have one person talking about a product's design and another person about its performance and a third person about the sort of community it's developed. We get one or another, and it's never organized.

I love reading about design, less so about pure opinion on subjects. Having multiple takes on tech would be a very cool feature that I've never seen on a site before.
but i would like to think there is a ethical point of profit (I.E. credit card companies in my mind lack a company '"ethical backbone")
I think your wish has been granted - conditionally. If you use haskell, you have QuickCheck, which is quite amazing.

If you use scala - you can use ScalaCheck which is essentially a port of the same ideas to scala. Very cool stuff.
I think "veterans" should be a little more understanding; realizing we were all one time a "newbie". I would even suggest we realize that a "newbie" isn't necessarily a newbie at everything; most of the time they can bring a valuable skill set to the community.
So maybe it's not that great (that's not what I had heard), but you have different providers!! That is at least a huge step up from what we have here. That is like music to the ears of an American. Here you pretty much have no choice except, Cable or DSL.
seems cool; I like the idea of having code I can copy and paste and "magically" features appear on my site. The problem is most of the time "magic" doesn't work out.
He must have been a symbolics founder like Dick Gabriel? (Dick now does the IT conversations stuff, and seems like a good guy to advise and mentor a new generation, except sadly we don't seem to want to learn from the past in this industry).

Personally if I meet someone who was around in the 80s, spent time with xerox machines, or (sometimes) at some great lab/company, I plant myself, and hoover their brain dry. I can't get enough of software history, there is gold in their experience. I just wish it was more common.
The economist and the financial times started recommending renting a few years ago - and they acknowledge that this was a remarkable turn of events.

That said, I think that deliberately renting is a short term play - one that paid off for folks who stayed out starting 2004 or so. These people will probably be in a stronger position to jump in later (though it's incredibly hard to know exactly when... and I suspect this bust will go on for a long, long time).

That said, when I was looking for apartments in SF a few years ago, I met people over the age of 50 who were losing their rent controlled apartments. They were truly shocked at the prices, and were angry about it. Of course, I'd never actually come out and say this, but I thought it: "you've been in SF for 25 years, was there really never a good time to buy?"

Maybe not. I grew up in SF, and over the course of my life, housing has gone from quite expensive ( early 70s) to very expensive (late 80s) to brutally expensive (early 00s). I actually think that a lot of people just never quite managed to get a toehold, and I feel bad for them.

Personally, I'm making payments on an expensive 1200sqft house with a little yard in an unfashionable corner of the city. It may well be worth less than I paid for it, but I locked in a 30 year fixed loan, and I'm not struggling to pay the morgage (though it is a big chunk of my pay).

When I bought, I actually would have predicted a downturn as well. But I don't really act on those beliefs - any more than I get into and out of equities based on where I see the stock market going. My best guess is that housing has a lot further down to go (something I believed as early as 2002 or so) - but my guess is no better or worse than what any other astute reader of the business section could come up with. But I'm a lot more confident in my belief that when I'm 55, I'll be glad that I own my hous and that I'm not worrying about whether I'll lose my rent controlled apartment.

I still think that starting early, investing in stocks and owning your own house, is by far the winning long term strategy.
Twittering the earthquake in China
Young Girls Not Interested in IT Careers Due to Lack of Female Role Models, RIM Study Finds
http://dowser.sf.net

A desktop metasearcher + archiver + personal search engine. It got lost in the noise when Google & Yahoo had their pissing match over desktop search in late 2004. The failure wasn't inexplicable: it was just too much work for one person... which makes the quote on the bottom of the page bitterly ironic.
No one in a position of importance said this demo was a google killer. It's not, it's basically a technology demo and hopefully a usable tool.

If you want a more fair comparison (you may not), site restrict your search to wikipedia and then compare. Both powerset and google don't do well here, the closest they get (and they both get nearly the same answer for the first few hits) is a mention of the tenth Vice President. I'll tell you why:

Tables. The data for this is in "List of Presidents of the United States" in a purely tabular format. Tables suck for everyone right now! They're not really NL, they're not keywordable, they're basically a nightmare because they require a pretty surreal amount of context to grok.

This isn't really an excuse, I admit. Powerset is working on it, but we just couldn't have it ready and still put out something when we wanted. At some point you have to put out something to show the world, and most powersetters feel the current tool is interesting and helpful. It's not perfect, or even necessarily complete, but I don't think anyone with an informed opinion expected it to be a google killer.

But seriously, if you're going to do a comparison at least go apples-to-apples and site-restrict to wikipedia. And yes, today at the lunch table the talk was all about what new data sources to grow into next, and it's publicly available information that we want to scale to the entire web.

Half the point of a pre-product like this (and lots of sites start out with reduced-power versions of their final product, even Google) is to just get the infrastructure made. Anyone who's got half a brain can tell you that getting infrastructure to handle non-trivial search is a huge task. You have query analysis, you have indexing, you have parallelized index servers, you have failover, these days you have EC2, you have databases and backups and monitoring and performance analysis and... arg. It's a lot of work.

And that's _done_ now. Powerset has a real infrastructure serving real queries to real people off a real index which is updating in real time (as fast as we can afford to do it right now). This isn't the end of Powerset's effort, it's the beginning. I wish people would take it as such.
Give them a break. They tried, and are still trying, to make search better. Hell, they launched. That's more than most of us here have done.
Feel free to try again, site restricting to wikipedia. ;)
Why is Yahoo boosting domain fees by 30%?
Well said - I remember reading somewhere before that the database was mentioned (as usual) as the problem area.

Someone else in the comments mentioned 400K messages a day, plus personal messages, say thats even another 400K.

I have zero experience working on a high traffic web application, but I work on a serious database application were we can push somewhere in the region of 1M transactions in a 6-8 hour window. Each of these transactions comes in the form of an XML string and results in as many as 20 plus selects and maybe 10 - 20 inserts.

Our application is build on top of Oracle and implemented in PLSQL - not too sexy, but it seems to get the job done.

The point I am trying to make is that with a bit of a caching layer it takes some serious throughput to reach the limits of database scalability (certainly with Oracle, where is where my experience lies).
I guess nowadays such things are made as Firefox extensions, e.g. Breadcrumbs: https://addons.mozilla.org/en-US/firefox/addon/2954 .
> first tries at something big

That'll be the Second-system effect ( http://en.wikipedia.org/wiki/Second-system_effect ).

If you've previously had a manager who vetoed your doohickies then there's a risk that some get implemented by your startup. Any of them could be a scalability nightmare.
Dowser is/was amazing. When I was at Flock I would always use it as the example of what we should be doing with search in the browser.

Mozilla's start page with the google box is nice but nothing compared to dowser.
I never saw this when you first posted it: http://news.ycombinator.com/item?id=149257

Note that I left those out of the Archive :)
How green are biofuels?
That's... actually very cool and gratifying to hear. Thank you. :)
I studied in Washington, DC during the last four years where the campus was rife with students interested in politics, non-profits and finance. Maybe it was my school, but I didn't meet any "techies" until my Junior year, and that was only a handful.

The article points to a reason why I left DC post graduation: technology, any type of technology, has some relation to politics. It wasn't the right place for me.
Because that's how things work in a lot of other services?

For instance, if you sign up for shared hosting and go over your alloted bandwidth, they may charge you $0.50 a gigabyte or so. That is linear and it seems reasonable.

What would happen if you could stream high-quality video from mobile phones? With large data transfer capabilities would come interesting new possibilities for markets, companjies, and customers.
MySpace does have massive scaling problems, most directly related to the difficulty of partitioning a relational database.

http://www.baselinemag.com/c/a/Projects-Networks-and-Storage...
Product design is as much a part of their marketing as their commercials/advertisement. The strength of Apple is that they convey a single message to the world. It extends to advertising, product design, store design...all the to the way steve jobs carries himself in his keynotes. Everything that they do says "minimalistic design with focused functionality."
Are We the Dumbest Generation?
Yep. When the Slogger plugin came out I decided Dowser's time had come & gone.
Ask YC: Transfer ownership of a domain name?
I'd love to hear any advice on the best ways to transfer ownership of a domain name-

A domain associated with our company is currently being used as a personal blog; We made an offer, and agreed on a price to buy it out, but I'd appreciate any advice on where to go from here.

The easiest way I can think of to alter the ownership is to simply send them a check, then set up a quick GoDaddy (or similar) account just for the transfer.. We could tell them the password, and keep a copy ourselves.
Once ownership is transferred to the transient account, we could then change the password and move it to own own accounts.

This seems a bit kludgey, of course. There are a lot of steps where things can go wrong, and there's a lot of trust involved.

I see that Network Solutions has a policy of getting notarized documents faxed back and forth, but we don't want to overwhelm the guy who runs it.. The chance of him abandoning the deal because of difficulty is greater than him abandoning it because of malice.

Has anyone transferred a domain name before? Did you use a 3rd-party middle man? What were the pitfalls or pleasures of the journey?
http://cadmium.avecora.com/

Failed social network advertising system that targets advertisements to Facebook Application users, by profile keywords (interests, activities), age, gender, specific networks, etc. Also targets ads throughout the web in participating partner websites.

Apparently no interest, and then Facebook launched Social Ads, with similar targeting features a month or so later.

Was hoping to integrate it with other social networks (Bebo, MySpace, Hi5, etc) and integrate with other websites to serve highly targeted advertising.

Also, ask me in a month, and I'll see how well my current venture is going. :/
A search engine for Wikipedia?

I can’t believe people actually finance such projects.
ICQ didn't archive a copy of every message sent.
Ok. The results are almost the same, except that Google's results have a much better UI. While Sarkozy was fourth on Google, and third on Powerset, it was both much higher on the Google page thanks to the ridiculously huge Powerset "facz" box, and it was easy to scan because there wasn't so much visual clutter vying for my attention (like yellow highlighting everywhere).
Previewing Google Friend Connect: Website owners can make any site social
I don't know if gasoline in 20oz containers would be any cheaper than water.
Social-networking sites work to turn users into profits
We don’t need more information or aggregation, we need inspiration
Just get the auth id from him, make sure he unlocks it and initiate the transfer from the registrar you like. Name.com makes it easycheese and is also $6. I have 300+ domains on there (all transferred the same way, either owned by me or others).
http://tagger.flaptor.com - automatically suggests tags for blog posts or articles.

It's still mostly a demo, we are improving it very slowly because most of our resources are going into other projects.
It doesn't imply that. Owners and renters have different risk profiles (or ought to). In financial terms, most new homeowners are *highly leveragedC in real estate. Unless you are a skilled speculator, this probably represents more risk than you intended. Landlords absorb this risk, and are compensated for it.
Great sense of web design on all your sites, and being 15 is pretty impressive too :-)
http://nshrine.com/ A place where social networking communities can share shrines, memorials, etc. It still has a chance, but its growth is pretty disappointing.
See http://news.ycombinator.com/item?id=129079
Good analysis of Powerset by Danny Sullivan
Kickass! When you paste the Disclaimer text into the box, it suggests "the darjeeling limited" and "potty training". The first three paragraphs of Yegge gave me "faithfulness" and "registry cleaner". Just make it work, and it will be awesome.
It does depend what you mean by the overloaded "web services" term.

If you just mean in the sense of a generic API (restful or otherwise), then for c) I'd include defensio, and then I use all the API from bug.gd and featurelist.org (because we use them a lot internally)

I think widgets and the like will be using these a lot more often. Right now, we tend to develop everything of ours as web API first and then grow them into widgets, apps, Flash, ajax on top of that as needed.

Saturation is a harder question-- I can't say that I'm sad at all that more companies/initiatives are opening-up APIs.
Careful with GoDaddy. One of our fairly innocuous link request emails (yeah I know, I know) got reported to Godaddy and they threatened to re-absorb our domain. Fortunately our sysadmin defused the issue. I can't recommend an alternative as we're still at GoDaddy because our higher-ups don't take domain asset management as seriously as they should.

Most of my personal stuff was registered through doteasy.com, and I have no complaints after 2+ years with them, though these sites typically do light duty, YMMV etc
There's also the issue of the timescale. Start with one programmer and server. You have the finances to purchase 10 additional servers or hire one additional programmer.

You can get 10 servers bought, delivered, installed and configured in one week. These servers can be deployed to do anything from load balancing, web caching, database caching, being database read slaves or application servers. Regardless of the quality of your architecture, 10 servers will probably add some scalability to your system. Furthermore, they can be re-purposed as the situation demands.

Hiring a better programmer (or just another warm body) takes longer. Furthermore, the improved code they produce isn't as flexible as surplus hardware.

Of course, the situation changes when you've got thousands of servers.
Apple trademarks iPod's shape
"Even a website that users visit daily seems to feel like a tool: users want to get in and get out as quickly as possible rather than “connect” with the site."

This is great advice and makes me think of the recently failed Nau - http://nau.com - they tried so hard to make their web site "engaging" that they wound up making it impossible to shop there. If they had a really simple site and Threadless-quality emails things might have worked out.
Thanks, but that doesn't talk about the nuts and bolts of actually doing the transferring, which is what I'm interested in ;)

We've got the negotiation and agreement part down, but I appreciate the link none the less!
-Colin
Np. I put a link to http://ask.searchyc.com where it said "Ask YC".
I love your wording: "Keep This Shrine Ad Free" :-)
BlackBerry launches $150M "BlackBerry Partners Fund"
Not Knot, "a guided tour into computer-animated hyperbolic space." (video)
Sell now for $Millions or holdout for $Billions? What a great problem to have
Ya know...I feel for you, but right now you're a victim of your company's own PR. If Powerset hadn't been blasted through everyone's media filter as a Google-killer, I don't think anyone would care that your first product is tiny and incomplete.

Expectations management is important. That said, if you guys have the resources to persevere, this might not matter. Ask Microsoft about the Zune....
I am having an impossible time coming up with a unbiased and substantiated opinion on this issue. On the one hand, we know nothing about Twitter's architecture other than they're (to some extent) using a language that has a notoriously under-optimized interpreter and a framework with reported scaling issues (unless you do a lot of hacking to it).

On the other hand, there's gotta be something fishy going on in Twitter-ville. Although I agree "the idea that building a large scale web application is trivial or a solved problem is simply ridiculous," by now Twitter should have enough performance data to know exactly which part of the process is causing the high-load issues they're having. If we are to assume that after each outage they at least "throw more hardware at it," then, theoretically, it's a problem that horizontal scaling cannot solve and the issue is deep-rooted in the system -- somewhere in the basic architecture of it.

Is it Ruby? RoR? Poorly optimized queries? Improper caching? Lack of domain knowledge? Leprechauns? I don't know, neither does anyone else outside of Twitter... but I guess speculation can be entertaining.
With all that money my son would have done better!
New Visual Search Engine Picitup Needs to Pick Up Its Game
That's fair, but it was the first time I'd seen it. I found it enjoyable and insightful, and I'm not exactly new to this internet thing.
What To Look For In A Business Partner
Its funny when I think a site like about.com have as much if not more page than wikipedia. What a financial blunder!
http://free.LocalDataPlace.com - which we built for local recycling (ala freecycle). We haven't been willing to spam existing freecycle groups but we need good local traction to catch on.
Yes, it doesn't know when to shut up. It tends to work better with longer articles, we usually test it with content from Google News.
You Should Learn New Languages
"No one’s going to beat Apple by being "good enough". The only way to beat the iPhone is by creating something better."

Uh, beat in what way? He can't be talking about selling more units. Seems to me Nokia et al are still winning by being "good enough".
Well that depends on how you pronounce FAQ. As soon as words like that get colloquial, they are pronounced as one word instead of as a list of letters (fak vs ef-ay-que).
To summarize the article "By choosing to compete on design instead of technology alone, Apple seems to have found a loophole in the Innovator's Dilemma."

Can we apply the same type of concept to Internet based company and be successful? Does 37signals use this concept?