skysight-inc/HackerNews huggingface dataset:
I'm also wondering why this kind of intervention is necessary at all. The NoSQL solution we use at work has load based automatic splitting, and I'd have thought (though I haven't confirmed) that this would be an obvious feature to include.
As a technical guy who is taking care of the biz side of things in my startup, I must disagree with you lachy. I know how aggravating it is when someone shows complete disinterest in the technicalities. They lack an awareness of the challenges and think this stuff is magic. It's not. There are limitations and trade-offs.

I'm learning a lot of mechanical engineering stuff from my co-founders that's critical to understanding why our product will or won't work.

As far as passion is concerned, you should be passionate about the whole thing - from start to finish! I want to know the why, how, and what behind my products.
Worthwhile and funny.

Q: What's your viral strategy?

A: We don't want anyone to know about it.

Still laughing.
If that's true, it was a really stupid move...
The problem is that pot-stirring, half-wrong stories, and superficial snap judgments bring the eyeballs, inlinks, discussions, and ad revenue.

Ars Technica has managed to hold the quality fort quite well, so it's not necessarily a doomed enterprise.
A system to push/pull selected messages to distributed servers (securely) would kill, and would be the basis of a revolution around the internet. I think Diaspora botched that chance, even though I sent them an email about it. Maybe they didn't really, but I hear their code is bad.

Besides, we need a platform, something like CouchDB, Diaspora is more of an app on top of the platform (but maybe I'm wrong on this one as well).

In a nutshell, easy/secure push would be a killer platform to build the new Twitter/Facebook/FourSquare on.

Of course, this couldn't be monetized easily, so maybe I'm wrong.
This has been well understood since antiquity (in the CS world at least). Read Jim Gray's "The 5 minute rule" and the more recent papers that cite it. Most likely the access frequency of your objects is not such to demand them residing in L2.

Ultimately there's no need to use a commercial database either, as there are compelling open source alternatives, though if your needs are very specific, a commercial database may be your best tool.
Which NoSQL store do you use?
MySQL is not written in a very portable manner. I prefer PostgreSQL for a number of reasons, and the fact that its developers are concerned with portability is one of the more minor reasons.
Thank you, I was hoping it would be good. What's rooting? Is it like jailbreaking? I thought Android phones were open?
Thank you, I hope it'll be a nice change from the uselessness of iOS...
True!
Usually people run around looking for co-founders and they look like people saying, "Build this for me, and hey you are a co-founder that means I don't pay you. Your time is at risk not mine."

And there is no way they can get a co-fonder that way, anyways.
I don't see how Ubuntu's benefits over Solaris are relevant to a comparison with FreeBSD.
Most of them aren't as broad as the titles indicate (often the case), though they don't exactly seem like significant technical innovation either. The first two are part of the infamous set of FAT patents (http://en.wikipedia.org/wiki/File_Allocation_Table#FAT_licen...).
Without knowing more about their schema, it's hard to speculate. However, let's assume they had a single, large "checkins" collection, sharded by user ID as they said. That would be distributed across the shards. If the mongod process is down on one shard and therefore the shard isn't responding, the data just wouldn't be available.

But it sounds like the shard was still up. As per the sharding FAQ[1]:

"What if a shard is down or slow and I do a query?
If a shard is down, the query will return an error. If a shard is responding slowly, mongos will wait for it. You won't get partial results."

They said they had a performance issue on the overloaded shard so perhaps it's possible that mongo believed the shard to be up, when it was instead overloaded. This meant any queries just waited, taking down the whole site.

As I said, this is speculation and as a massive production user of MongoDB ourselves[2], I'm interested to know more.

[1] http://www.mongodb.org/display/DOCS/Sharding+FAQ
[2] http://blog.boxedice.com/2010/02/28/notes-from-a-production-...
The old guard is losing power, their business models are failing, their control is going away.

A new dream is starting. A different dream than the fantasy of being "discovered" by a major record label and becoming rich and famous overnight (which translated into a reality that often didn't quite match up to the dream, for the bands or for their fans). A new dream where increasingly many artists are bootstrapping themselves into making a living from their music without big record labels interfering with their creative process, sucking the bulk of the profits away with little in exchange, and adding layers of corporate bullshit between artists and fans. Instead, bands are connecting directly with fans, selling music directly to fans, producing their own albums on their own schedules and releasing them however they like for whatever prices they wish to charge, and making the lion's share of the profits for themselves.

This is happening right now. Spreading right now. Becoming the new way that music is produced and distributed. It's something to look forward to.

The music industry is dead! Long live the music industry!
Note 3: Don't do this if, after a week, you don't like coding. You'll go insane.
Absolutely true. I was making the point that they've survived and are apparently thriving. However: Pandora and similar services have had to work far too hard to do such a straightforward thing.
The entire point of redis's VM layer is to serve data sets larger than ram when the request distribution is such not everything need be memory resident (I believe this is sometimes referred to as the 1:10 problem in the redis community).

While ram is far cheaper than it once was, there still are substantial savings in reducing your resident set requirements from TBs to just hundreds of GBs.
Ironically, the servers are being "upgraded" again, as we speak.
http://labs.google.com/papers/bigtable.html
I like Digg better. But if you have that many people on board, better go for something innovative. I like how L-pad does it: http://lpad.in/mentors
USA is rapidly deindustrializing and there is no
necessity in damn science kits for kids. Kids must
know what is credit card, lawyer, insurance and mortgage.
And what is barista and security guard.

Americans. Do not invent new measures. Look what Europe (or even Australia and UK) does and just follow the wise.
Instead of excessive tests for science kits it better if you ban unsafe components (such as azodicarbonamide and trans fats) from bread, ban corn syrup, stop importing apple juice from China, or some other, REAL things.
It's not a trial. It's refusal or follow a court issued order, the punishment is contempt of court. They don't need a trial in that case... the judge that issues the order can simply judge whether or not you're complying with it.

In all these scenarios, you're screwed the moment you find yourself in court (actually probably long before then when you convince some law enforcer that you're doing something wrong) Judges and lawyers aren't going to be hip to this hypothetical plausible deniability game.

A drive isn't an extension of your brain or your relationship with your wife. It's physical evidence. The only thing that makes it different than say a really big lock on the door of your house or a safe is the effort it takes to circumvent when a court agrees to admit the evidence. Almost never is that evidence used alone.

There are subtle issues to the meaning of the evidence provided, like you might not need to give up the password and thus decrypt everything else encrypted with it, you might just need to decrypt the data in question in a convincing manner to the court. Being able to decrypt the drive doesn't have to prove that you are responsible for the contents.
Idiot VC questions - hilarious animation
You can get commercial support from 10gen (MongoDB developers) as we do for my company. It's just like MySQL providing support for their enterprise DB. Of course MySQL is much more widely used than MongoDB so there is more community help and knowledge.
Outlook is MS's preferred utility for storing contacts. Syncing your contacts on a winmo phone means importing them into outlook, essentially.
Wouldn't that take a very long time to re-encrypt a whole drive with a new key?
How is LRU not simple? The page replacement policy in Linux is simple it's just makes use of available information - The algorithm does not make the system. Clock-pro is not in Linux and I doubt it ever will be. The goal in creating manageable code is passing the information that you know to the next person that doesn't by using comments and even Linux doesn't do this at all so hack-ability of any non trivial parts is near impossible without a few months of poking.
But it wasn't the overloaded shard that brought the site down (it was only slowing things down), it was the addition of the new shard:

"For reasons that are not entirely clear to us right now, though, the addition of this shard caused the entire site to go down."
If your a hack what are you doing reading hacker news? jj :-)
Yes, of course I read it. Yes, the allocators are synthesized; so what? The point is that "synthesize a semi-customized allocator for each program" is a very different thing from "just use what the system provides you with; you won't be able to do better".
"Fear is the mind killer" is from Dune. http://en.wikiquote.org/wiki/Dune#Dune_.281965.29
I loved the comment about the upgrade treadmill. Some build systems have unintended consequences.
What do you mean 'just' a subjective state of mind? Fear indicates a fact - you have learned a reason to be afraid in situation X.

"Situation X makes me afraid" is a repeatable, testable statement. It's not coming from my consciousness so it is plenty objective enough. You don't feel it, but what difference does that make?

If ignoring it worked, if "just do it" was good advice it would be so obvious, such common advice that such a blog post would not be necessary. If just ignoring fear worked, fear wouldn't have any power over any of us in the first place.
I like the NoSQL approach as an option. But we should keep in mind: operationally, these databases/stores are comparatively new, and don't have the years and years of use that would help find and solve problems like this. It reminds me of Ebay's 3-day downtime in 1999 -- based on an Ebay mistake and an Oracle bug. Although Oracle had been around for a while in 1999, OLTP was still new, and, hence the bug.

I'm not blaming a flat-out bug in this case (the cause of the severe part is still unknown?), but it could also be architectural or operator error.
Check out this comment:

http://news.ycombinator.com/item?id=1761410

Now observe that the FTC's complaint references email from Jason arguing with the rest of his team that they should continue doing this, because the account represents "all the margin" for his company.
I've started flagging any articles behind a paywall because I assume that it's the correct course of action. Please tell me if this is wrong.
What I don't understand is this: Why do companies (like four square, twitter, etc) wait until after their first multi-hour crash before instituting a "this is how we'll communicate to users when we have downtime" process? I would assume that everyone has learned from twitter's historical mistakes at this point. I would argue that startups -- especially startups that deal with large numbers of transactions per day -- start with code and policies for communicating downtime issues first and launch the product second.
How to Get Him Back: 3 Ways to Help Keep on His Mind
When your boyfriend breaks up with you, there are so many emotions flying about and things running in your mind.
Learning Rails – or not as the case may be
Can't you call a Truphone user from Gizmo, since they both use SIP?
> I’m sure whatever product manger they have wants me to do something more than anything else. Maybe that’s completing my profile, accepting friend requests, seeing sales and offers (hello revenue) or simply searching for a restaurant.

Actually, it really does look like Yelp wants her to complete her profile -- and search for something in San Francisco. Everything else is kind of a wash, as pointed out, because the page is too damned busy.
Affiliate booking revenue
It's a post from my own blog, but I'd really be interested in what HN thinks (are any others in the same position or have they been and how did you get out of it)
Google Goggles released for iPhone
Your landing page looks great, but if you want to increase conversions even more you should try putting the subscribe form above the fold. It took me about 5 seconds to figure out how to subscribe.
I got one, it's a popular add-on to last will packages from document mills like http://legalzoom.com

Where I am, the local hospital asks for a living will before any kind of surgical procedure (as well admissions that could turn into one, eg. childbirth), so I am guessing quite a few people have them.

I don't know whether this is typical or just local practice.
Maybe you knew this already but if anyone else is reading here is a really simple example of the like to interact code - http://thinkdiff.net/facebook/how-to-detect-fan-of-a-faceboo...
If the data was being moved it could cause even more load on an already overloaded shard.
Windows monetizes like crazy, too. As did Blockbuster. As do highway robbers.

Companies can monetize at the expense of goodwill all they want. It doesn't impress anyone but their accountants and investors.

Meanwhile, users grow resentful. Companies with a user product should value the user first and configure their business accordingly — it's just the only viable long term play.

If you do crappy things to the user for short term cash, that becomes your long term play because you become dependent on the money. And in the long term, you lose to the guy who doesn't treat users like crap.

Edit: >All sites start out idealistic, and then start making compromises in order to survive.

And if Zappos had taken this advice and compromised by screwing their customers for cash, would they have sold for a billion dollars?

I mean, a billion dollars, all for not being a dick to users or cutting corners at their expense. Thats the value of ideals you won't compromise.
Right, I'm aware. Not everybody can afford a contract though, nor should they have to in order to avoid major outages like this.

But I would like to know if Foursquare has a commercial support contract with 10Gen. If they didn't, why not? especially for a service that big? If they did, how was it that 10Gen took that long to fix the problem?
Tell HN:  Experience-based pivot for the musical industry
In a nutshell: computer-generated hologram artists realtime A/B testing musical performances using the audience's mood as a feedback channel, maximized for enjoyment.

So, you start with some seed-music -take an upcoming (but yet unlicensed) performer, or an old music where the license is already expired.

Add some randomization, instruments, mutation, pay other artists, what have you.

Point is, do real, but 100% software-based performances on stage, and start A/B testing it against the crowd.

It kinda snowballs from there; because while most artists do test, their performance-hours are massively limited by the expectation of their physical presence.

It's 1, scalable; 2, non-replicable-able in home enviroments; 3, unique experience

Note: I'm not in the position to take action on this idea -just want to get this off my brain. Feel free to steal, copy, or improve :)
I think the simplest way to dodge those patents is not supporting long file names.

And it's really weird to have patents issued over technology in use for almost 20 years and charge licensing for it.
Read here: http://viewtext.org/article?url=http://www.nytimes.com/2010/...
Heh, I went to historify that site and post the cache to help out, but wasn't sure if it would be seen as shameless promotion. I'm proud this feature sees usage, and glad you like it!

EDIT: I'm glad to report that server load is at 0.01 right now, so hammer away, Varnish is pretty much indestructible.
Y'know, us developers are also "ideas guys" in addition to being developers.
In my case it seems to reinforce my dislike of Twitter.
Not so great, since they expire after a few minutes. Remembering isn't that much a part of the service.
> And frankly, I can only code for so long before getting bored, and losing interest.

That's because it's not all fun and games. It's hard work, and the most important part of an internet startup.
I disagree. Time is money. But which opportunity cost is greater? Do you have access to more time or do you have access to more money?

If your primary skillset is in business, marketing, or sales, stick with that. Don't deviate from your specialization.
This sounds naive. The people running the labels are very cynical, and the way they're compensated makes them even more so. They don't have equity. They're motivated by bonuses that come out of this year's revenue. So they're not interested in building stuff. They just want to extract large amounts of money, this year, from whoever they can.
It's amazing how symmetric but decoupled the problem is with good people looking for great companies and great companies looking for good people. Adam Wiggins from Heroku described it like dating sites: in theory, there are plenty of people of all sexes looking for other people to date, copulate, or marry -- but in practice getting to a match is quite hard. We might have been talking about startup financing at the time, not hiring, but there the situation is similar. Is there a name for this category of problem?

For me, networking really is an under-appreciated skill for engineers. It makes finding something awesome much easier. Also, it seems like the best thing I ever did to increase the size of my network was do a startup. You're constantly talking to people about financing, biz dev, sales, or even just commiserating / celebrating with other founders.

Dropbox is looking to hire some great people. Dropbox is an awesome place to work with one of the best reputations in tech, which is great. But, it is still hard to hire. [obligatory self serving link: http://dropbox.com/jobs , ask me anything: ivan@dropbox.com ]
You can get this from dropbox, now, for free.
I think true sales is 'putting your head down'; cold calling is at least as manly as debugging Ruby. But I think "biz-dev" is often, well, spinning bullshit fantasies, putting them into powerpoints with nice colors, and calling it work. Not always, but often.

I guess that's why proven coders and proven commission salespeople (and proven accountants) can always find jobs, while "big picture" folks are left unemployed, wondering why companies can't see their brilliance. Or, if they're lucky, they sue Zuckerberg because he was able to do make an actual product which never got past the great-idea-over-beer phase for them.

(Ducks...)
Peter Theil, co-founder of Paypal, would probably disagree. In the book Founders at Work, you'll find that many co-founders of large, valuable companies claim that having another co-founder understand the business side was invaluable. (Note: I'm not trying to stick up for them because I'm a business guy; I'm a coder guy.)
It's not clear whether this applies to urban speeding or highway speeding.

Urban speeding does increase accidents because of the large number of unexpected events in an urban setting (pedestrians, cars driving into intersections, etc).

But the main issue I have with speed limit studies is that seem to make this underlying assumption that everyone follows them. If the core group of accident-causing people drive fast/drunk/etc regardless of the rules, then lowering limits/changing rules must have limited effect.

Underlying all this is the government love of speeding taxes. The only problem is, the marginal return on speeding tax decreases as the tax take goes up. Because people decide to obey the limit, the tax take starts to go down. Thus the conundrum for governments - do they set the fines at a level where people are annoyed but don't mind, or do they increase the fines to the point where revenue steeply drops.
i don't honestly understand what is wrong (to the extent what it is almost considered a 10-letter word) about stereotypes usage. A stereotype is just a result of analytical and/or statistical aggregation. In this particular case he didn't stated what all Russians are shady. He stated that there is a [non-empty, and we all know it is really far from empty] set of Russians that have common trait of being shady and there is subset of them such that some email spamming work can be outsourced to them. What is wrong with such stereotype usage?

And is it just my bad luck with Russians (what are the odds?), that i do prefer Moscow-Paris flights when majority of passengers are French than the flights of Russian airlines with majority of passengers being Russians?
Thus science proves it.

Whisky is the cure for the common cold! ;)
No reason, go for it. Seems like fun. I don't think you'll ever change the "musical industry", but a city bar would work wonderfully.
Poll: How many HN users also use Quora?
Just curious to see what the overlap is between HN users and Quora users since both have great but sort of similar content / people.
I use Quora.
Have an account but never use it.
Don't use Quora.
If I had a co-founder with that attitude I'd tell him to sod off.

The people at the top have to be willing to do whatever it takes, whether they're "passionate" about it or not.
For sure, I guess some people are 'full-time-idea-guys' in the sense that they just develop and manage them full time.
Load based splitting is on the MongoDB roadmap, but doesn't exist yet.
and they had a road-side billboard near my house recently. Clearly going after the IE6 demographic :)
Ha-ha, oh, wow.

http://en.wikipedia.org/wiki/Uses_of_torture_in_recent_times...

http://en.wikipedia.org/wiki/Torture_and_the_United_States
I think that's inline with Derek Sivers history (as told on TechZing) http://techzinglive.com/?p=443
You will NOT become an engineer, programmer, or web developer, but you will be able to put a prototype of your idea together and maybe get one or two beta users for feedback...

You will also be able to have an intelligent conversation with a developer.

I get sad whenever I encounter a business person with no technical bullshit filter. Not because I'm judging them, but because if I can bullshit them, any other developer can. Which probably means there's a problem somewhere that will hurt all of us.

I'll make a point to have a cursory understanding of financial statements, market segmentation, and project management if you do the same for the basic building blocks of software applications. Then the two of us will be able to talk about almost anything. OK?
Is it the next 10 mins. The next 2 tutorials that you're after?

Always considered the first 5 mins pretty well documented, but then after that you're in a requirements driven area, of I need my app to do xyz. How can I get it there.

I guess my thoughts are what are you trying to achieve with your next tutorial in Rails? I'm guessing you've found RailsCast?
Does your local hospital provide you a Living Will form?
I submitted the article - It's not behind a paywall. I even cleared all my cookies and I didn't even hit the registration wall...
Show HN: Coding noob - first weekend project
A few reasons:
* It's open source so cheaper + no vendor-lockin + it can be built for a wide variety of purposes vs. a monolithic single purpose app.
* It's extensible. You'll be able to download a wide variety of Eduglu-compatible apps built either by me or other people/companies.

It's geared toward social learning. Where we learn by observing and talking to fellow-learners.