skysight-inc/HackerNews huggingface dataset:
The post doesn't mention the environment in which you're running Cassandra. Any chance you're running it in the cloud (EC2?), or are you running it on real h/w?
Hypothesis: if you find that you have a balanced life you'll eventually get burned out on that too and shift the balance.
I know Twitter's just done the dirty with Cassandra, but it seems Cassandra is getting a massive PR boost this week, despite being one of the older (where old is 2008!) NoSQL systems.

Can anyone explain/posit an idea as to why MongoDB and CouchDB have been stealing the thunder for the past year rather than Cassandra? It just seems odd.
The article points out that it has already scaled with hundres of kilobucks or even millions.

However, it also points out much lower-end hardware solutions that cost under $10k but perform much better than the largest EC2 instance, for I/O.

ETA: This is why I tend to roll my eyes at the notion of "commodity" hardware. The article's low-end array is 400MB/s, but rolling ones own can yield over twice that for the same or lower price tag. All this well before reaching the unscalable cliff of enterprise pricing.
It took Zuckerberg more than five years to make Facebook the #1 web site... he's still not quite there but probably by the end of 2010.
Google Responds To Privacy Concerns With Unsettlingly Specific Apology
Brillen mit Rezept Online Kaufen
Website bietet ein Netz von Optikern, nach Ihren individuellen Bedürfnissen für Ihre Brille, Rahmen der Wahl der Produkte, einschließlich Sonnenbrillen, regelmäßige Brillen, Kontaktlinsen und andere Produkte mit guten Qualitätssicherung.
Selling Services for A Fixed Price
I'm actually at the same stage, so I can't offer too much advice, a few quick things.

1) I like your design, company name, branding, etc. well done

2) I'd re-write 'marketplace for datasets of the Web's structure, like the entire link graph (in chunks, of course)'. You didn't describe it that way in this post, you just said 'datasets from ongoing web crawls'. That is WAY easier to understand. I'd use that as a starting point. Just tell people what it is that the site does, or more important, what it can do for them.

3) along with the previous remark, your keywords can use some work. use the google adwords tool https://adwords.google.com/select/KeywordToolExternal. you've got 'datasets' when in fact, 'dataset' has 10x as many searches. I'm sure you can get better descriptors as well.

Keep up the good work!
Thanks. The cross-platform nature of shoooes is a lot of work, and it's been holding up the 1.0 release for a while.

It's looking like a Shoooes 3 release may happen soon, though, and that should help. I'll certainly let you know when it's usable.
This idea is really more of a tautology. It takes of-order 10 years time to become an expert at something because this is of-order a human lifetime. There is no objective "Expert" level. "Experts" are just the people who have spent the most time at something (modulo effort and intrinsic skill). Since (1) you can't spend much more than 50 years on something and (2) there are always diminishing returns for time invested, 10 years will get you 90% of the maximum possible skill at almost any pursuit.

The exceptions are, of course, those areas where there are aren't strong diminishing returns (rare...maybe memorization?) or where there is a ceiling on skill (e.g. tic-tac-toe). In those cases, expertise comes with more and less time, respectively.
Sure, rigorous proofs are often obtained by "proceeding formally", but there's usually (not always, granted) some more intuitive reason as to why something should be the case, otherwise nobody would have tried to prove them in the first place.
Interesting. I don't think I agree with this example, though:

He might, for example, ask students to recite the letters A through J as fast as possible, and then the numbers 1 through 10. Each of those tasks typically takes around two seconds. Then he asks them to interweave the two recitations as fast as they can: "A, 1, B, 2," and so on. Does that take four seconds? No, it typically requires 15 to 20 seconds, and even then many students make mistakes.

That's not a fair comparison. When I quickly recite the numbers from 1-10, I'm not figuring out which number comes next each time, I'm reciting an incredibly well-rehearsed script I have memorized. Same with the letters.

I could memorize the sequence A-1-B-2-... Then I bet I would be able to recite it in 4 seconds.

To better capture the "switching time" that purports to demonstrate, you'd need to interleave two tasks which each require that kind of "next step" generation. e.g. Time someone giving every other letter and the square numbers, for instance.

It's pretty easy to determine the sequence for each of those, and it's unlikely that someone's memorized the whole thing already.
I love Scoble's response to this:

I'm more likely to check out Divvyshot because of this post. Thanks, off to check it out!

Any publicity is good publicity, it seems..
All of your replies here suffer from the same logical flaw and many people have pointed it out. Fog Creek has a policy of not divulging their reasons for denying an applicant a certain position. Joel has given his arguments about why that is sound, so I will not bother to repeat them. You keep assuming that Fog Creek should know that it would be safe to make an exception in your case and getting offended that they choose not to. The fact is you should not be an exception.

To make this as clear as possible, suppose you were trying to get pain killers without a prescription. No matter how obvious it is you're not a junkie and are in pain, it would not be appropriate for the pharmacy to sell you the medicine based on their subjective judgment. It may not matter in this instance, but it would undermine the whole system.

The reason why I respond is because it has been frustrating for me to read your replies on this thread because this gap in reasoning has made all of your comments sound very personal. Anybody can sympathize with the frustration around being denied a job, but your story says nothing about Joel's management approach.
Reading this article was a bit annoying. Right from the start: "I work in the financial industry [...] I worked in the insurance, telecommunication, and power generation industries." I was thinking only - you're not even supposed to look at nosql from that perspective - there's nothing for you there... just go away.

It doesn't support transactions and acid most of the time. There's no "we pay $xxxM for the support and blame you for everything" company in nosql products. It's not the same workload as you'd expect from kv/document-store used as a webpage backend.

One of the few serious "nosql" databases for enterprises like that is Berkeley DB - it's got what they need. I'm not sure why did he write that blog post... it just stated the obvious, but in form of a rant.

The funny thing though is - Berkeley DB is exactly what NoSQL is about... and it is used for local reliable storage in many big enterprises. Replication, logging, transactions, etc. - and it's just a kv-store really.
why don't you open both the links in tabs? that's what I do. If I want to comment, the page is already loaded and ready to go. If I don't, then I just end up closing two windows instead of one.
My favorite part of the rant is the suggestion that the nosql alternative to spending millions on a ginormous RDBMS was throwing away throughput by using Amazon AWS. I guess any argument becomes easier to make when you define the other side by its incompetence.

The only valid point made seems to be that vertical scaling of a RDBMS can be a multi-million dollar exercise.
The point is that world-class runners from countries in which most people train barefoot have far fewer foot defects than people from rich countries who are engaged in the exact same occupation.

What does {{fact}} mean?
Bloom Energy, fuel cells or fool’s cells?
This was really brought home when a fellow entrepreneur mentioned the other day that Google makes it near impossible for startups to enter a category in SEO.

I have to chime in, because this is just plausible enough to be dangerous without actually being true.

Incumbents have several built-in advantages, however, that doesn't mean startups can't compete with them. Practical examples are using your vastly superior access to the linkerati (if you don't know what this word means, go read SEOMoz, it is probably the most-important-least-obvious thing about SEO I could name), willingness to experiment rapidly, and lack of structural impediments to engaging at scale.
Google Product Ideas: Tell us your feedback for Buzz, share your ideas on Buzz
How do you know this? Is there any real data on this topic?
We need to blackhole every single one of the 55 users who upvoted this submissions. This is precisely the kind of stuff that will turn HN into digg. What's next - lolcats? Seriously?
>The company has been roughly doubling its revenues every year — 2007 came in at $150 million. We expect that trend to continue for the foreseeable future, making Facebook a multi-billion dollar company within the next few years. The question is becoming how Facebook can hit the inflection point where its revenues increase much more quickly.

Um, doubling every year sounds pretty good. Can they, or any company that's hit that size and revenue level (it's easy to 10x or 100x your revenue when starting from near 0) really hope to do much more than double revenue yearly?
And yet: http://i.imgur.com/3HM89.png
Although some proofs can accumulate cruft.

E.g. there was a small and nice intuitive prove of a reasonable theorem in the first place, but scores of people found ways to generalize it, and in the end you have a mess of symbols proving a hopelessly general lemma.
This is a very reasoned, well-written article. While I like the idea that the NoSQL movement is questioning some core database assumptions, I have often been uncomfortable with the meme that "SQL doesn't scale". Scalability is always a function of your circumstance. This article makes a good point that the group of people for which SQL doesn't scale need another option (internet startups with little cash and massive server load). While that case gets a lot of attention around here, it's important that people not try to extrapolate that experience to other areas of IT.
In the pet-lizard case, aren't you using the phrase with mild irony? Something like, "After painstaking analysis coupled with a few brilliant flashes of insight, I managed to deduce this totally obvious fact"?
I've read a surprising number of resumes from people who seem to bounce from company to company wanting to be the idea guy or gal. I don't know where they got the idea this position exists. Maybe it's from some bygone era where giant dinosaur companies needed to institutionalize change by bringing in people who were explicitly there to "shake things up".

Everyone we hire had better have a "shake things up" streak in them. Thinking without doing has no place in a startup.
We are running in a virtualized environment, we've done a pretty good job at adding capacity for performance and space reasons ahead of the demand. Loadbalancing is pretty hands off, its just something that takes some time and fundamental understanding, as there are still ways to shoot yourself in the foot with this system.
If you came in blind to Loren on this particular episode, sure, he comes off as a bit of a tool, but he's playing this as theater. I don't know 100%, but I think he's playing a great "angry man" persona (in a similar way to how Zed Shaw did for a while - just in video form).

I rarely agree with anything Loren says, but he makes for compelling, funny watching. Take his recent review of ChatRoulette, for example: http://youtu.be/nMlTuS39zC0 - watching Loren is like visiting one of those theme restaurants that have deliberately bad service.

He also looks like my plumber, so that makes him pretty cool in my book.
Apologies; I hadn't had issues with Scribd myself (Chrome/Linux), didn't realize others did.

It is also on slideshare, and I can upload to other desired formats.

It is originally a Microsoft PPT, as that makes the 15 seconds timing easy to enforce and practice.
Hello world in VVVV
The Smell of Entrepreneurism in the Morning
Something has always bothered me about companies like Facebook and Google that generate such a high percentage of their revenue from advertising. They strike me as quite vulnerable.
Anyone want to explain that diagram of the while loop to me? The author apparently believes it is so self evident that he doesn't have to explain what any of the symbols or colors mean.
maybe SQL scales, ACID probably don't
I studied computer engineering in college too, and I'm a woman. It wasn't a huge deal, and there was some extra attention sometimes that was nice, and other times was not nice. But there were also small things like that the computer engineering dorm floor was male only or when I got a microsoft internship and my lab partner said "must be nice being female" that made me feel different.
Is this guy a genius or just crazy?
> "between August 1994 and August 2004 [...] Citigroup alone has returned a breathtaking 848%

Because of their excellent management, right...
Picnik: Innovation - Giving Birth to a Startup
Ask HN: Question about startup salary
I've been offered a position in a bay area startup (10th employee) and since this will be my first opportunity working for a startup, I'm looking for some advice. The equity package is reasonable, but I'm not so sure about the salary. They're offering about 75K - I have abt 4 years of s/w experience, albeit in a different technology. Is this a reasonable offer ?
IPhone apps, iPad apps, Android apps.
"Such a platform can yield very satisfactory performance for tens or hundreds of thousands of active users"

There are 253 million Internet users in China alone. What happens when your site needs to scale from 0.001% of them using the site simultaneously to needing to scale to 1% of them using the site simultaneously? Within a month?

"Of course if you index poorly or create some horrendous joins"

Which in the Twitter and Facebook cases is exactly what they have to do on many of their requests. As I've personally found out, relying on a database to do a join across a social network graph is a recipe for disaster. One day you'll be woken up because your database's query planner decided to switch from using a hash join to doing a full table scan against 10s of millions of users on every request. Then, you'll be left either trying to tweak the query plan back to working order, or actually doing what you should've done in the first place: architect around message queues and custom daemons more suited to your query load.

"Even with billions upon billions of help tickets."

At 50 million tweets a day, Twitter would hit 18 billion tweets within a year. Good luck architecting a database system to handle that kind of load. That is, one in which the database system is serving all of the requests (including Twitter streams) and isn't just being used for data warehousing.

"Such a solution — even on a stodgy old RDBMS — is scalable far beyond any real world need"

The disconnect here is this guy's needs are not the needs of a lot of us who are actively looking at alternatives. He is simply not familiar with the problem domain.
Sometimes (often, even)! But with that particular example, it's actually more like, "I didn't ask the questions that many second-year medical students might ask after finding out a patient had salmonella."
I would wager that SQL, and ACID, will scale larger than many startups will ever actually need.
Microsoft sees a new road ahead, with the same old speed bump
Consistency, Availability, Partition tolerance. NoSQL stores usually sacrifice consistency, and instead settle for eventual consistency. SQL (i.e. RDBMS) stores, with their usual emphasis on transactions, must necessarily sacrifice something else. SQL that doesn't hew to a hard line on consistency and transactions doesn't really have all the features of SQL. This is the distinction that matters most, IMHO, in the NoSQL strand of thinking.
Intraconnectivy
http://techcrunch.com/2009/09/15/facebook-crosses-300-millio...

While I didn't find anything newer (in about 20 seconds of googling), facebook was profitable, and I see no reason that it wouldn't still be so. If someone feels like putting in the effort of finding newer information which proves this wrong, please do; I would love to be proven wrong (well, not really; but if I am wrong, I would like to know about it).
Prepaid EC2 — a simple way to cut amazon contractor cloud costs
On track to win my bet with jacquesm:

http://news.ycombinator.com/item?id=689993 :)
> daily "aha" moments, that i cant get from books at this frequency.

You’re reading the wrong books.
how to read more books: pirate them. or buy them online or to an ereader. however you get them, convenience is key. i read 2,000-3,000 pages a month since putting books on my computer. love it.
definitely crazy.
http://www.rebelscience.org/Crackpots/nasty.htm
This is his page where he claims all of physics is wrong.
Except for postgresql.
Months ago, I attended an entrepreneurship talk at which the speaker argued that the textbook and journalism industries would die as the scarcity of these media ended.

One week ago, I attended a talk by an editor at one of the major US publishers. He said they'd been searching for a way to modernize but didn't have a response to my mention of the previous event.

Looks like the type of problem with plenty of money for new models, much like the music industry today.
You're never going to convince anybody of anything if you can't explain yourself without using jargon. To be fair though, it looks like he's written a lot on the subject so cherry-picking one piece out of context is going to cause confusion. The guy should write some kind of manifesto that anyone can understand.
"that’s not an argument at all! It’s a blind assertion based only on my own experience"

And this is a problem, making assertions/arguments based on direct experience?
Uh, crazy, I think. From his "all physicists are crackpots because nothing can move in spacetime" page (http://www.rebelscience.org/Crackpots/notorious.htm):

"Why is motion in spacetime impossible? It has to do with the definitions of space and time and the equation of velocity v = dx/dt. What the equation is saying is that, if an object moves over any distance d x, there is an elapsed time d t. Since time is defined in physics as a parameter for denoting change (evolution), the equation for velocity along the time axis must be given as v = dt/dt which is self-referential. The self-reference comes from having to divide dt by itself. dt/dt always equals 1 because the units cancel out. This is of course meaningless as far as velocity is concerned."
Reading aloud is fun, and can be useful, but it is extremely slow. No one reads 100 books aloud per year.
I wonder how many people in the NoSQL and SQL doesnt scale crowd either have never met a truly competent, much less good DBA (trust me, they are very very rare) or decided it could not scale because they applied their programmatic and procedural logic to a tool that operates in a very different (SET based) paradigm.
It's easier to slap an injunction on a company shipping boxes full of patent-violating product.
http://www.rebelscience.org/Cosas/components.htm#Types
how does this play with the thought that if an idea doesn't find enough traction in 3 months you should move on? i get the impression from investors and startup bootcamps that they want fast results seemingly more than commitment. how many times have we been reminded that startups completely change their ideas? how do we balance flightiness with steadfastness?
I'm all in favor of alternate models of computing, or styles of programming, but this guy writes like he invented cold fusion or N-rays. I think he relishes his self-appointed status as an embattled minority/shunned Messiah.

This is not to say that his ideas are totally without merit. I'm sure there are problem domains well-suited to his approach. However, just like all previous One True Ways, it's unlikely to be always be the best way of doing things. He also glosses over important details like his claimed hardware support, and how you're supposed to construct non-trivial flow diagrams in practice.
William Safire's weekly "On Language" piece in the Sunday New York Times did this type of thing for decades and was required reading in my English classes. Sadly he passed on recently but you should be able to find stuff like this in his books or in his articles in the NYT Archive.
Read this recent thread/article and reexamine your equity position:

http://news.ycombinator.com/item?id=1151743

Also, read this:

http://www.tonywright.com/2008/a-newbies-guide-to-startup-co...

What is your % stake in the company? If you aren't getting a good piece of equity then I would push for a market rate salary based on your position and years of experience.
Precisely. This sounds like an argument for the superiority of static typing more than anything.
Should I learn PHP or Ruby?
The 64-bit compiler I wrote for LoseThos is for an improved C/C++.

void DoIt(...)
{
int i;
for (i=0;i<argc;i++)
coutln argv[i];
}

The supid varidactic syntax of C was replaced with variables argc and argv[] that automatically exist for variadactic routines.

You don't have to label a "main" routine. You invoke a function by a statement outside of functions.

Where do I begin with all the improvements? http://www.losethos.com/doc/Differences.html
I admit that the SQL vendors (besides MySQL) made a mistake by putting ACID above scalability; that's clearly not always the right choice. However, CAP still allows a SQL database that is scalable, consistent, and available.
This is a perfect response: thank you.

Also, the definition of scalability seems fairly clear: how many times can you square your capacity before you need to re-architect?
No, that is not a reasonable offer. It is a typical offer, though.
thanks - i'm getting around 0.3%
Is it possible to call someone on this in conversation without being rude? e.g. "What do you mean 'it turns out'?"
Who all on HN is at MIT right now?

Despite MIT's reputation for entrepreneurship, it's been difficult to find collaborators actually willing to take the start-up plunge--perhaps especially in grad school.
I'm curious how services like Amazon's RDS will change this perception.

A SQL database may be difficult to scale, but it is something that can be largely encapsulated and outsourced. If Amazon RDS, or some other product, handles the hardware and software configuration, then the developers can just focus on the application portion of it.

This isn't to say that scalability is guaranteed, it's still important to optimize queries and the data structure. Also, there are problems where NoSQL is simply the better and/or cheaper option.

But if these services can encapsulate a lot of the difficult part of scaling SQL, it still makes SQL a very attractive and powerful option for most(?) problems.
Still doesn't really help. What's the while loop actually doing? Where's the data being stored, and how do you get it out?

Similarly for his quicksort example (http://www.rebelscience.org/Cosas/quicksort.htm) - from what I can see, there's no means to input or output whatever you're sorting - that part is all internal, and the only accessible parts are 'start' and 'done'.
It is only a bad thing when they become exclusive about it. And that is bad for all the same reasons that the way they were treated in high school is bad.

Speaking personally I grew up without a TV, get motion sick if I play first person shooters, and don't like sushi. Despite being a stereotypical geek in many ways I can't count the times when I've felt excluded by the fact that I don't know Monty Python inside out, have avoided most games produced in the last 15 years, and don't want to eat at certain restaurants.

I can only imagine how much worse it can be if I didn't catch and appreciate other geek references.