skysight-inc/HackerNews huggingface dataset:
Perhaps you could use FastBit ( http://sdm.lbl.gov/fastbit ) along with mysql. This paper may of interest:

Analyzing Enron Data: Bitmap Indexing Outperforms MySQL Queries by Several Orders of Magnitude. Kurt Stockinger, Doron Rotem, Arie Shoshani, Kesheng Wu

Abstract
FastBit [1] is an efﬁcient, compressed bitmap indexing technology that was developed in our group. In this report we evaluate the performance of MySQL and FastBit for analyzing the email trafﬁc of the Enron dataset. The ﬁrst ﬁnding shows that materializing the join results of several
tables signiﬁcantly improves the query performance. The second ﬁnding shows that FastBit outperforms MySQL by several orders of magnitude.

http://www.osti.gov/bridge/servlets/purl/881612-KQlHWx/88161...
Regarding "competitive intelligence" you could check out RivalMap for some ideas, which is a cool looking service.

http://www.rivalmap.com
The other thing to do is to look after the bottom X% in terms of absolute poverty - making sure that they have basic food, medical, and so on. You could still have a lot of inequality, but at least no one falls beyond a certain point.
or perhaps only being able to downvote a story once it reaches X points, so it doesn't get downvoted to hell but doesn't stay up too long either
I posted an Ask PG submission about the use of downvote.

http://news.ycombinator.com/item?id=214398
Ok someone needs to come up with one for "how to hire a web designer". :)
I totally agree with the above comment, but would also add that most start-ups are really tight and small groups... who also happen to be way more susceptible to turnover. The best thing to do is to be yourself, and make an honest judgment if your culture meshes well with those around you. Meshing together is often a much bigger deal than experience, at least in my experience.
Rambi: German Car/Motorcycle Hybrid [video]
A new flavor of Google Trends
Steve Jobs WWDC Keynote (in 60 Seconds)
SharePoint To Run Enterprise 2.0?
"Isn't there some way to improve the incentives on issuing the first few votes?"

Making the "new" link on the header bar much more prominent?
He's right. It isn't that Jason said it, it is the way it is stated. Words have tremendous meaning, subtle and overt. It does come off snake-oily.

"Build your own Internet Business in 24 hours and make millions like our own affiliates. Just copy-and-paste this simple link and we'll deposit the cash in your bank account before you wake up! In fact, we guarantee that you can walk into work tomorrow morning and tell your boss I QUIT!"

Over the top, sure. There are a million variations to say the same thing and they all leave a different emotional reaction to folks who read it.

I don't think this is going to change people's opinions of 37Sig, but it is a useful lesson in being very careful with your words.
Symantec Buys Online Backup Service SwapDrive For $123 Million
But what happens a year or two after that? The best answer to the question, “Will computers ever be as smart as humans?” is probably “Yes, but only briefly.”

what does that mean? that it then will be much smarter?
I do think we have more interesting things to talk about here.
Be honest with the co and yourself. Self select who you interview with make sure its where you want to be. Large corporate jobs can bounce back from bad hire easier that a small co. Don't snowjob them, be honest, and make sure its a good fit. If so, work as hard as you can to get the job, and bust arse like never before once you do!
1024x768 is #1, but the surprising fact is the use 1280 x whatever is now #2 (instead of 800x600)

Thank the laptop/LCD boom for that.

I mockup up all my designs at 1000 pixels wide.
Short answer: yes.

Long answer: it's just a changing user base. I won't elaborate in this comment though, just that's an article in and of itself.
Do you really care that much about your karma here? Just ignore it, it's just a number. It's not a video game.

Obviously, the most interesting, useful comments are often going to be controversial. In cases where people downvote stuff they don't like (which is a proxy for, does not fit my worldview), these comments will not get the most points. This means you shouldn't care too much about points except that they tell you where the conventional wisdom lies.

Personally I downvote something if I think it's dumb. Since there is more stupidity here now, it seems, there are more downvotes. That's good.
don't be "weird," that's just weird. be yourself. i say this because when i was younger, i had pride in being "weird." at some point though, i realized i was just dicking around with what other people thought of me. it's a waste of time and energy. though it may have been a necessary phase for me

i guess i used it as a "warning" to other people. like "hey! i'm weird! that means if i do something awkward, it's no problem because it's expected!"

now, i am just myself, and too bad if my persona bothers people ^_^ it's a straightforward no-bullshit type of thing. free
Could this read any more like propaganda?
The problem is that downvoted comments are less prominent- they're all the way at the bottom of the page (and greyed out). Thus if people start downvoting things they just disagree with, things that contribute to discussions won't actually be seen, reducing the quality of discussions. Karma isn't the problem.
Oil shortage a myth, says industry insider
Defense contracting is the most hideous inefficient case of taxpayer robbery in existence today. I am not saying it is unnecessary, but the politics and waste behind it is astounding.
"If I disagree with someone, I won't vote up. If I really disagree with someone, I'll post a rejoinder comment."

I haven't been here very long but I've thought that expressing disagreement isn't the purpose of a downvote at all. Instead, it's meant to indicate that a post detracts from the discussion and probably shouldn't have been posted to begin with. Using downvotes to express disagreement would only result in discussions becoming more intellectually homogeneous, which would make the site much less valuable.

I'm not sure how the site can encourage people to use downvotes more appropriately, but I think that there is a higher risk of having trolls pollute the site than of downvotes deterring legitimate discussion (has anyone felt like leaving the site or not posting a comment because you feared that your perspective would get you down-modded? I'm not aware of such an instance but if it has happened to someone than that is a big problem that should be talked about).
do not feed the narcissist.
SocialText unveils Socialcalc, the first social spreadsheet
Definitely depends on the audience. If you know your site/app will be used on older machines or by less tech savvy folks, it's often safer to stick with 800x600 in those cases. Otherwise, do some assessment and decide based on that.

If there's no reason to exclude people though, then you may as well include as many as you can. Older browsers (IE6 anyone?) can be cut on newer things as well if the time to test and ensure it works in them isn't worth it, but that depends on your audience again.

Then there's the accessibility question. You're not developing for users within public or govt institutions are you? ;)
The biggest issue is whom you choose to interview with. The worst case scenario is a false positive = you get hired and it is a bad match/you don't like the job. This isn't a corpo job where you can hide or distract yourself all day.

So with that in mind be very honest and only look at projects/companies that you are genuinely interested in. Don't reach for any position.

Make sure you like the co-workers and do some research on the product and the market.
Agreed! It's amazing how many applicants you get that have only ever written code for school assignments or other jobs. Contribute to OS, start a project, just do something visible! It also shows you actually like what you do, which startups are definitely looking for.
From my experience, most companies want someone who will add value on day one and who gets along with the existing team.

Don't be intimidated by the questions. Keep your confidence high and make sure to answer honestly in a straight forward manner.

Good luck.
> Democracy says that this place is peopled with hackers and if the hacker citizens upvote a story then it is, ipso facto, of interest to hackers.

1. Not all hackers share the same interests. Just 'cause some hackers upvoted a story doesn't imply that the story is of interest to (1) hackers as a category, or even (2) the hacker community found at Hacker News. You're conflating two uses of the word 'hackers' with your "ipso facto": one referring to a specific subset of hackers, the other referring to hackers as a whole.

2. Mobocracies are just democracies characterized by unruliness and impulsiveness: it's perfectly feasible to have a mobocracy composed entirely of hackers. You don't need demi- or quasi-hackers to cause the problems we've been seeing, and it's not clear that this is in fact the cause for deterioration in relevance of submitted stories. I wouldn't be surprised if most of it comes from hackers who just aren't being thoughtful about what they submit.
This suggests an interesting way of looking at the problem, namely "How do we make the site less interesting for non-hackers?"
Life Insurance For Old People
Best Free Health Insurance Quotes For Individuals
I try to make my layouts scalable (measurements in ems) and fluid (the least fixed dimensions the better) to avoid problems. I'm pretty confident my (unreleased) site should be able to handle less and more than 1024x768 which is my resolution (if I wanted to scale down to mobile devices I'd make a version of my site specifically optimized for them).
"When you are busy trying to sell your technology and not the emotional impact it'll have on me -- you are missing out on one of the most powerful human motivators."

I think marketers rely on emotion a lot to sell...
Gun to head, yeah, I do care about karma. But I only care insofar as I respect the community who's voting.

The problem is that the indiscriminate downvote homogenizes discussion. The consensus opinion rises to the top. This is good when the group is good; bad when the group degrades.
Or that the advent of ai will cause us to wipe each other out
i discovered the 'new' link several months after i joined. the reason is because i thought new stories automatically appeared on the front page, and would simply sink fairly fast if nobody upvoted them. but it turns out they first arrive in some sort of purgatory from whence they must be uplifted
I've been down-voting some comments because it seems like everyone is up-voting any old comment. It's ridiculous for someone's "I liked this article" type of comment to be up-voted.

I thought up-votes for something special, to be given out once in a while as a way of saying "your comment affected me in a deep way, thank you". If you give out up-vote willy-nilly then no one really cares about writing a thoughtful comment.

Here's an example of bad up-voting: http://news.ycombinator.com/item?id=213070
What the fuck's the point of the up-vote there? Does dangoldin really need to be up-voted 6 times to indicate that his comment was useful? Wouldn't 1 up-vote be enough?

Anyway, I need a coffee and it's just a number.
I have found that, for example, comp.lang.lisp is active, vibrant, and helpful.

Although, a lot of discussion seems to be taking place on IRC (freenode in particular) these days as well.
It's okay to design with 1024x768 in mind, but you have to also consider those who are using handheld devices, screenreaders, and don't forget the bots. Make it good for the most common, but don't let it break for the rare.
YC:Looking for Suggestion
I have clone something this website with features like category and search box. I am not sure which industry or what to do with it. What industry will benefit from it. What area should I focus it on. Looking for suggestion. Please help suggest ideas or partner?
I got downvoted for saying it's not proper etiquette to passive-aggressively downvote. I just chalk it up as one of those stupid things you come across every now and then and move on. There's other more important stuff to worry about.
This has the honour of being the first story I'll downvote... Assuming I can, and it lasts that long.
With the rise of the iPhone and other mobile/desktop browsers, this seems more important than ever.

What does everyone here think of creating a separate .mobi type site versus taking the time to make your regular site "mobile-friendly"?
Navbars are probably the most widespread ui element on the web, and I can't think of anything you could do (that isn't explicitly malicious) to confuse casual users more than removing them. Even if that violated expectation counted for nothing, I'd estimate that 30% of the dental site's users wouldn't think to use the browser's find button. I would fire this man in an instant.

Even if we didn't have to contend with the habits and training that users have, I doubt this would be better. Is he arguing that a navbar with links to the appropriate sections is not worth the space it takes? If not, is he arguing that the benefits of everything being on one page (more useful browser find, I guess) outweighs the downsides (seo, users wading through mountains of text, scrolling required to access navbar, etc)?

On the other hand, check out songza.com for some smart one-page web antics.
IPhone is about to get a whole lot more social (Loopt mentioned)
Do Subscriptions Really Save You Money?
The RESTful way to do that would be something like:

POST /transactions

from: /accounts/savings
to: /accounts/chequing
amount: 123.45

Generally the proper way to do something atomically is to do it in a single request.

REST isn't the best choice for every application - the whole point is that it has certain strengths - but it's very flexible.
Switching to the FreeBSD kernel seems like it would have been a good idea back in the Rhapsody days, but Tevanian probably wouldn't allow it. By now Apple has added so many OS X-specific features like IOKit and DRM that it would be a lot of work to switch. Costs a fortune to maintain, costs a fortune to switch.
Our current website is comfortable at ~800 (I believe the content is 780 pixels with some elements that overflow), but the next iteration will be designed to be comfortable at ~960.

Just remember that really wide text is hard to read, and break it down into a readable column width for text, plus other elements.
I've had my own encounters with the downmod problem (we all have) and I won't repeat them here in the interest of brevity. But here's a suggestion for handling it. Can't tell if it's pie-in-the-sky, so input's encouraged...

Most of us make it a habit to abide by the news.YC "guidelines," such as not changing the text of submitted headlines unless it's needed for clarification, and not adding comment signatures. And it's an easy thing to police because there's accountability. Case in point, I've only seen a signature once, and another user had pointed to the users' misstep by the time I noticed it.

But there's no accountability for downmodding. There really can't be, else you injure the chance for honest opinions from people afraid of the kickback. But what if there was a single line of text you needed to include with a downmod? Something short, like 25-40 characters, that explained your reason. The community would recognize anonymous feedback like "Trolling" and "Unnecessary personal attack" as reasonable motives for downmodding, and "Bad point" or "Stupid idea" as unreasonable.

It wouldn't eliminate the problem -- I don't think anything anonymous could -- but it might create some mechanism for policing indiscriminate downmods and encouraging fair use.

Of course where you'd display that information is another issue entirely...
I suppose coming up with something as successful as Facebook would be a nice problem to have, but isn't Facebook's origin a little questionable? Zuckerberg's partners were obviously unable to protect their idea but if kynikos is already working for an internet company there's a potential problem pursuing an area they could be interested in.
I just started interviewing developers for Tipjoy. I don't think there is anything in particular different about a startup. We want really smart people. I really like to hear about projects done on the side, when no one was looking and you didn't have to do it. You should check out Tipjoy, and the other good YC companies who are hiring: http://news.ycombinator.com/jobs
That's probably because Emacs is such a nice NNTP client...

/me ducks for cover.
I think if you give someone the option to downvote, there are always going to be people who do whatever they want with it. Telling someone not to downvote if they disgree with a well-presented argument is like telling someone not to post crap in the first place (or not to upvote crap, for that matter). Most will abide by the guidelines, but there's nothing you can do to stop the minority from misusing abusing it. Unfortunately, the wrong minority is enough to ruin the feature (or even the community, although I don't think that has happened here).
What do you mean by "passive-aggressively" downvoting? Do you mean downvoting for disagreement with a subject? Because I agree: that's a poor use of downvoting.
i saw that comment you made, agreed it was downvoted spitefully, and upvoted you in compensation.

people are getting way too touchy with the downvoting.
To justify a parallel, separately-designed .mobi site, it takes a lot of traffic from handhelds AND serious design reasons to make the main site low-resolution-non-compliant (loreznoncom! I made up a Web 3.11 word!).
This just sounds like a Wiki with a different interface. It's obviously collaborative but I don't get what makes it "Social". Are you supposed to invite your friends to your spreadsheets or something?

Hey everyone! Party on at the 2009 budget!
it's not that simple. to participate effectively here, you need a certain amount of karma. you can't downvote without it. and if you go into negative numbers, you're almost certain to get your account terminated.
Since bank-robbing is such a big problem, I suggest closing all banks. I also suggest preventing planes from flying with passengers inside, as they can possibly get killed if the plane crashes. The odds, in fact, of getting killed in a plane accident are probably bigger than the odds of the NYAG finding child porn in a random newsgroup posting.
For me the Youtube embed wouldn't load on the site but http://youtube.com/watch?v=dGY28Qbj76A works great.
Here’s a compairson of Powerset, Google and Yahoo:

what are the symptoms of dementia?

http://s281.photobucket.com/albums/kk207/dementiaquery

Notice that 60% of the results on Yahoo and Google’s pages are sponsored links that show no immediate information. The Powerset result is much more useful.

That said, the Powerset output would be little better if it used the full width of the window.

One thing I really like about the Powerset results is how you can click on the arrow to the left of the result to open a more detailed summary of that result in a small text box, allowing you to scan the results without losing context.
I second this. When startup jobs are bad, they're really bad.

(good corporate job - bad corporate job) < (good startup job - bad startup job)
I notified http://infochimps.org/ about this. It's a cool place to get data sets, check it out.
Thanks for all the responses! One piece of information I did not mention is that I am not an engineer/hacker but a biz dev/sales guy. I don't think that changes much but just thought I would put it out there in case it does change anything. Thus far I have been very selective about the startups I have chosen to interview with. From the advice here, it looks like I should continue doing that.
"It's because programming is more like painting than
engineering."

I think pg would agree.
FriendFeed openness puts pressure on rivals
Twhirl Supports FriendFeed Rooms, But Not Seesmic - Quite Yet
i agree this is happening, and that it's a problem.

i think the karma threshold for downvote ability should be raised to, say, 60. if you can stick around for that long, you're more likely to understand the community's values.
The community would recognize anonymous feedback like "Trolling" and "Unnecessary personal attack" as reasonable motives for downmodding, and "Bad point" or "Stupid idea" as unreasonable.

Most of my downvotes are on comments which don't fall into any of these categories -- they fall into the category of "not even wrong". In my view, a comment which isn't even logically coherent has no business being here.
cable providers don't have long retention dates anyways. use giganews with newzleech to download stuff.
> Money "going away" isn't necessarily a bad thing. Sometimes it serves a useful purpose. For one thing, when someone's money goes away, they tend to go out and do more work, creating actual wealth in the form of economic output, rather than parasitically living off of accumulated money. Inflation is a form of money "going away." High interest rates are another form of it. One of the reasons interest rates are at a historical low, is the minting of credit money by banks like Bear Stearns. So, you can think of the choice as shifting the price of volatility (or risk; yes, risk costs money -think about it for a minute) from interest rates to banks. This sucks for guys who work for the banks (me) which have to manage this risk, but otherwise, a pretty good deal. Just like in the old days, the minting of credit money was a choice between not having an economy based on money, and worrying that you might wake up with worthless money.

Unless you're a banker or a government official, it's hard to agree with the writer that working hard in return for worthless paper is "not necessarily a bad thing" or that working hard and saving money in hopes of acquiring financial independence is "parasitically living off the accumulated money."
Building on the last line of parent's comment, you should also research your co-workers ahead of time. Spend an hour or two googling the company, their email addresses, screen names, etc. If you're decent with Google, you'll find a lot. You should be able to get some idea of who you'll be talking with and working with ahead of time, which, in my case, helped me to be more confident during the interviews.

Anecdote: I recently came home from an interview at a startup, one which shall go unnamed, and decided to do more googling of their employees. I found that they had been taking parts of my cover letter, which was admittedly eccentric, and Twittering them amongst themselves, mocking me. How embarrassing, more so since I had just been in interviewing with them. Jerks.
Here are some animations of the shell changing.

http://blog.wegif.com/?p=15
i can testify that getting mass downmodding for saying something unpopular was common about 9 months ago. it seems the same to me now.
This story should really have been sponsored by Jungledisk or Mozy...

Offsite backups! They're not merely a symptom of paranoia!
you should care about the reasons behind votes, not the votes themselves.
And I say does it really matter in the grand scheme of things? So you got downvoted in a virtual community...my gosh the mind reels.
It's not. Not easy - have you done anything else besides web development? VB6 is a far better tool for what you are saying.

Just because 99% of people here dream of becoming .com-millionaires doesn't automatically turn these fashionable "technologies" a best way to introduce a kid to programming.

The absurdity of misused web "apps" for commercial purposes is one thing (especially if you get paid at the end), but fucking up someone's education is just cruel.