skysight-inc/HackerNews huggingface dataset:
Diet and Fat: A Severe Case of Mistaken Consensus
You mean an openID like system that doesn't suck?
We tried this and failed miserably.
http://reddit.blogspot.com/2005/10/polyphasic-sleep-groggy-j...

My hat goes off to you for making it work.
/impressed
An interesting way to get around the problem of people lying would be to aggregate all the search words a single person searchs and see if there are interesting correlations to other singular points. You could write a plug in that feeds all your searches annonymously to a central database where you could run a gamut of tests.

To your example, when I get a new drug I look up all the side effects (so I would look up thalidomide). Then I have a child and it turns out that my child has birth defects. I would look that up to see who else has that out there has experienced having a child with the same birth defects. As search continues, it would eventually pop up that people who look up side effects of thalidomide also look up birth defects. Then someone could do a serious study on it sooner than later.

As in all cases using data, the major impediment is anonimity and scale of data collection.
Every science has its branch of engineering. Physics has Physical engineering, Comp Sci has software development, physiology has medicine.

What's the equivilent branch of engineering for sociology or social pyschology? I can think of: Propaganda/public relations/advertising, group and crowd control, realpolitik, etc.

Nothing too good in a general sense.
Wouldn't you expect a smarter society to, uh, trust people to make their decisions rather than leave it up to insane autocrats?

The most notorious insane autocrat of recent times appeared in the country with arguably the most impressive intellectual tradition.
A trend
There is a trend in Hacker News.

I remember when I first registered on YC news there were way less direct(no external url) submissions. Or so it seemed. This should be due to the nearing deadline for YC applications. It is vitally important to get as much positive attention from pg as possible; in fact it serves as one of YC news' main functions. But people should not try to be visible just for the sake of it. Sometimes the extra submission may just be the one that irritates YC.

Don't get me wrong, the majority of these submissions are great, interesting and fun to read. Although occasionally you see one of those and you can't help but wonder. What I am trying to say is that people should be sincere about everything they do, and if there is nothing good to say then just listen and learn. I believe that you will get treated by the Universe the same way you approach it.

But of course the real point of this submission is an attempt to show YC some criticism and analysis skills.
Interesting Non-Software "Startup" - Wave Power Generator
I'm pretty sure most everyone here is (like me) interested in software startups, but I found this link and thought it would be interesting to those with entrepreneurial bent. What I found so interesting is that the idea is so simple and easy -- why didn't I think of that!?
It also figures that the site goes down two seconds after I post this. Dern you Reddit!! ::shakes fist:: Here's a direct link to the company making the product: http://www.swellfuel.com/
We'd consider it if it's a great idea.

If it's a great idea, though, it seems likely it would
also appeal to the US market...
I'm a guy who's supposed to know all about TDD (I teach organizations how to run software teams) so I hope I don't jump the shark with this comment.

I just have a hard time seeing TDD in all cases. As one commenter pointed out, some systems evolve and some are specified. Evolving systems have execution units (classes, modules, etc) that can change a heckuva lot. Specified systems don't. On a specified system, you know within 80% or so what your hunks of stuff are.

Now I've coded a lot of specified systems. No, you don't design every little detail, that's not what I'm saying. But if you're writing code to balance a checkbook you're not going to end up with a program to make spaghetti. In the startup world, you very well could end up somewhere in a completely different problem domain. Whatever happens, I wouldn't want the testing structure to get in the way of the best design, simply by making it so hard t do heavy refactoring.
I think it'd be neat if Google Analytics let you send your actual log data to them. Even in an efficient binary format it would be a lot of bandwidth, but the accuracy and extra information would be worth it to me.
If you use thunderbird there is an add-on called Xnote that does just that.
I've had a number of friends who have been banned from facebook for what seem to us as the tinyest infractions (fake unoffensive funny middle name for example). They usually get a reply from disabled@facebook explaining why they were banned within a few hours. Perhaps email again? Be sure to be poilte and apologetic, they sometimes will reinstate your account if they're in a good mood.
> ... we realized that making something for public schools
> was a really bad position for a tiny startup to be in ...

Well, the question is: "Who is un(der)serviced in this market?" If the public schools are bad, it is probably the parents and students. I don't know much about your product, but one approach might be to see how it can be sold to parents who want to improve their children's education.
It isn't perfect, but I swear by GenPass for this -> http://labs.zarate.org/passwd/
you probably don't bond with people in any other job or activity the same way as you do in a startup.
I think it would be less bandwidth. Considering the overhead of all the individual HTTP headers both in request and response, you could reduce that significantly by batching up log files and uploading them at regular intervals, perhaps once a day? GA seems to perform batch updates at that interval in any case so it might not even put a big cramp in their style from that perspective.

However, there are a bunch more issues with this. To allow this, they'd have to be able to parse all of the most popular Web server log formats. This, in and of itself, is not that hard until you account for people customizing their logging format via measures like the LogFormat directive in Apache. As well, customizing the log format is a virtual necessity, as GA requires more information than is in the standard log formats of most web servers. Another issue is that the use of a JavaScript snippet on the page prevents non-browser user agents from appearing, clearing GA's logs of spiders and other such cruft that detracts from determining human user behavior (what they are apparently most interested in for the purposes of optimizing AdWords and AdSense).

But the biggest problem for them, I think, is that with this method they are no longer able to set first-party cookies on your user's browsers and thus (more) reliably track individual (read: human) users. So, in effect, they would waste time with this, in that they'd still have to make you put a JS snippet on the page to set cookies that you'd then have to modify your web server logging format to include in every hit. Urchin worked like this back when it was still alive and you could run it on your own servers. So, all in all, it seems like something Google would just be wasting their time in doing given what they want to get out of your data.
They don't appear to be relevant to Google because they don't help Google better determine ways to leverage AdWords and AdSense. GA's JavaScript snippet in fact returns to Google more and more pertinent information than is resident in the average web server log file. Plus it removes the spiders and non-page view data just by being a JavaScript-based tracking solution so that's less work and cleaner data for them (and anyone else who does this e.g. all of the other Web analytics vendors).
I don't have time for Hacker News anymore
I have submitted a massive amount of stories to Hacker News in the past, but recently I have been so busy, I don't even have time to read Hacker News, let alone submit stories...
I sure wouldn't be disappointed if I had shares in Urchin, or Google for that matter.

I'm sure all the Game Neverending enthusiasts were bummed too. But as a startup founder, I'm not going to sympathize much over how Flickr was a "disappointing story" for them.
Internet Allstars '01:  What happened to the most popular sites of 2001?
Bugfix: delete should work properly now.
I seriously doubt that. It probably came down to price.

Almost all apps that Google buys have to be rewritten anyway.
I can't get over a few things.

One, how many of these so-called startups are actually receiving outside funding? It's one thing to hack together a dumb idea and put it online with ad banners all over the place, and another to actually receiving funding for said idea.

Funding hackers versus straight-up entrepreneurs is smart on YC's part for a number of reasons, but most of all because sites like YouTube blow up just as much for the technology behind the scenes as the implementation of the idea itself. Justin.TV is viable not just for its idea, but the clever things they've developed to make live video streaming cost-effective. And this likely plays into why YC looks more for smart people, as opposed to smart ideas.
... but just enough time to tell everyone you don't have any time, eh? ;]
It is hard to give that detail without knowing detail about how many concurrent users are using apps. If we have more breakdown then it is easy to give that detail.

Without much requirement detail, On the safe side - My guess is three Linux server cluster (Specs 2-CPU and @4GB RAM ( @2500$) )should be enough to server content for high usage site like facebook.

It is purely guess based on my limited knowledge of facebook. If you have any specific detail like

- Avg Concurrent users
- Peak time concurrent users
- Apps software platform ( java app server, php, .NET ?? )

then that will help.
60+ Free Web Development Tools & Resources for Entrepreneurs
Physics of Hard Drives Wins Nobel
Ask PG: we don't have an idea, should we leave the yc application mostly blank?
how should we fill out YC application if we don't have an idea that we are really into? how can we solicit PG's ideas, and still prove we are hungry animals?
The subject of this article is two words too long.
Wearable Power Assist Suit
document.f.q.focus(); The Billion Dollar Line of JavaScript
Should you trust Facebook with your business?
YC Application question
I have submitted YC application. We have explained main idea in some detail and we have given one sentance detail on other ideas. How YC will get detail about my other idea. How they will judge my other ideas potential?

Should I create new application for other idea which I think has good potential??

Please help..
Build your own car... from scratch
Why Google GPhone Won't Kill Apple iPhone
I agree. It would be great if there were some way to filter the good submissions from the bad ones, perhaps with some sort of voting mechanism.
OK, I think you're missing the point of the "the idea doesn't matter much at this stage" assertion by pg.

I'm not the keeper of pg's mind on this stuff, of course, but I'm pretty certain that pg and Co. would like to know that you are capable of having good ideas--even if it later proves to be unworkable, unmarketable, or otherwise wrong. Having no ideas at all is probably a danger sign.

You have to be able to recognize problems in order to solve them...so if you can't think of a single problem in the world today that needs solving that the skill set of your team would be uniquely qualified to solve, then you're probably not quite ready to do a startup. You clearly haven't seen enough of the real work world, if you can't think back on a hundred and one things that you or coworkers did in inefficient ways because there was no good alternative.

The note about the idea being fungible is just that--it's early, and you may end up changing your idea through the program, or YC might say, "This is stupid, but you seem pretty smart, anyway...how about we sit down and come up with something else?"

YC wants you to be hungry _and_ smart.

In short: If you want into YC, I'm pretty certain you should find some kind of idea.
I've prepared my application ;)
And this, of course, is not an example of exactly that.
PG is an amazingly genuine guy and I think it is good to apply for those interested regardless of the result. Remember, the story of Reddit guys. Never loose an opportunity to learn from others and in this case, even by applying you will learn a lot from YC folks.
Your subject line was tempting enough for me to take the link, but then I was just disappointed by lack of actual content. With this kind of post, none of us will have time for Hacker News anymore!
Unrelated but relevant: YC Application Word Count

http://www.commonplacelog.com/ycbookmarklet.html
http://nymag.com/news/features/38951/

Topical, eh?
http://ycombinator.com/faq.html

You are allowed to submit another application under a different user name of one of your co-founders. But I suspect applications are judged just as much from what can be ascertained about the founders than the ideas themselves, so it might be a moot point. Good luck.
Good luck to all Winter '08 Applicants! 
As my mouse hovered over the 'submit' button, I felt a little like I had my finger on "the big red button." It was very tense for a few moments. Had we conveyed our ideas the best we could? Was everything proof-read for the 30th time? Could we come off as likable AND smart in under 120 words per question? Was I really that fond of living off Easy Mac again? After running through the mental checklist, I took a big breath in, and gave my trackpad a solid click. (spoiler alert: the submit button changes to a resubmit button).

So, I wanted to wish everyone who is in shoes similar to mine the best of luck on their application, and urge you to press that button and take another positive step towards building something amazing. The mousedown is scarier than the mouseup ;]
The ability to bury posts would be really nice.
This isn't quite what we had envisioned. This doesn't really do anything new...
Submitting stories is the easiest thing ever. I don't see why anyone would give up on that. Read something interesting? Two clicks and it's posted!
You must be able to think of something.

Just ask yourself "What are the major pains in the ass I deal with on a regular basis, and can any of them be solved with technology?"

It's probably better to go in with a high-concept, far-fetched idea than no idea at all. Just see what you can come up with.
Well, there's all this talk about applications being judged on the quality of the cofounders and the idea not being too important. A friend of mine applied this summer and was essentially turned down because of the idea (primarily because there was competition in the area). So maybe the idea isn't that irrelevant and applying with more than one idea might not hurt.
PG announced at SS07 that they would start considering applicants with no idea. I still imagine it's better to have one, though.
Well, that's one way to come up with ideas. The other is to think "what would millions of people like to waste their time with".
You are absolutely right, it may be a big red button!

Good luck.
We don't notice the userid of every comment and story, just long-term patterns. So it's probably too late for anyone to do anything (good) that would affect their application.
Thanks, good luck to you too!
This may have been considered already, but maybe things could be arranged so that downvoting submissions is allowed, but does not reduce the submitter's karma.
If this is accurate, it's pretty remarkable.
this is pretty cool but looks rather uncomfortable.
Video Service Showdown: Viddler vs. Vimeo
I've really considered YC News=Digg+Techcrunch+a Forum...not much more than that; I like to take info as well as contribute info to the community.
Touche'.

That's another good strategy.
Chicago Geek Guide Moves to GeekHeartland.com, More Guides to Come...
No. It mean that all the '.com' with generic names are taken.
I just put my other ideas into the box at the bottom where it asks me for "...other idea's you considered applying with..".
I am not so sure it is a "myth".

There is this old programmer.reddit post (http://programming.reddit.com/info/1kth0/comments) about an XP/TDD "guru" who does "exploration" with unit tests and another developer (a real guru this time, no quotes around the word) who doesn't.

TDD is a valid technique when it fits. What's annoying is this tendency on the part of some of its practitioners to make it out into some kind of panacea.
Thank you, that's nice. All the best back to you, maybe we'll see each other at the interviews.
Is this YC News' Birthday?
I randomly decided to see how much karma PG had (it's a lot more than the top of the leaderboard), and noticed that he has been a user for 365 days.

Does that mean today is YC News' first birthday?

If so, happy birthday! It's been a good first year.
He's right, Joe, we did. So if people want to leave the idea parts blank, that's ok.
Video Interview with Marc Andreseen (@25:00) The Gigaom Show
Related: Commentpress (built as a theme for Wordpress)
http://www.futureofthebook.org/commentpress/
Hmm, I suppose it is, in a way. The system has been up and running for 365 days. But 365 days ago reddit was in the middle of getting bought, and we didn't want to freak out Conde Nast. So the site wasn't launched till Feb 07.
My Wiki Can Beat Up Your Inner-Office Memo
Thoughts on why Web 2.0 tools are being adopted by even the big boys of business. Big companies, no matter how backward or resistant to change will eventally succumb to these tools simply for convenience. But what about Twitter? You'll be surprised...
I would be really curious about those dollars, and what percentage get an investment of 1 million plus.

6% seems like a 200% profit if each yc startup generates a 1 million dollar investment.

If the VC deal rate is closer to 1 in 10 or 1 per year, it looks like you need one $10 million dollar deal a year, each of which needs to have a potential of $250 million eval, or $25 million a year in revenue before the VC will be interested.

So my question is, if a team can demonstrate "real" projections with lower than average risk, and demonstrate a 2 year projection of greater than $25 million a year would you consider an offer of less than 6%?

My second question is, since we are in the Midwest, "What exactly is a rockstar?" ;-) We have Billy Corgan, and Garbage to be proud of, but the coasts seem to be more keen on the idea of the "rockstar," so, what are the characteristics of a "tech" rockstar?
I realize some misdirection on my part. I should post my ideas in this context too. This thread will be gone by Friday, so I'll continue to submit ideas as separate posts like these:
http://news.ycombinator.com/item?id=65320
http://news.ycombinator.com/item?id=65083
-------------------------------------------------------------------------

http://www.tipjoy.com/our2cents/2007/10/real_liveblogging_li...
"Real live-blogging, live comments, and a comment API"

Live blogging is an interesting aspect of the blogosphere. Bloggers post in real-time, often while attending an event, chiming in live with their thoughts and views. This is a great way for bloggers to communicate with their readers, but it's currently not well supported in blogging tools.

Live bloggers should be able to update their posts in some manner similar to a chat client, or collaborative editing software. The system would take the blogger's new content and appended to the post on the readers screen without a page refresh. It would be entertaining to watch something so live, along with a different take on the process of posting to a blog. Readers could see sentences edited and thoughts refined.

Ideas come from writing them down, and this would be a live crystallization of thought.

The stream could be synchronized to an event, and then later could be played back to get the same experience.

Similarly, comment sections could be live. This is nothing new, with group chat rooms updating messages to the whole group. But it would improve the user experience of a comment thread, with new comments appearing on the site live. For sites with comment modding, the votes could also be live -- watch trolling comments nose dive and disappear.

Regarding comments - comment systems should have open APIs. For example, if a blog post makes it to the front page of reddit, digg, and Hacker News, there will be 4 disparate comment threads, one at each of the news sites, and one at the original blog. Rather than forking the conversation, recommendation sites augment it: grab from and push to a comment feed. Comments on Hacker News should be visible on the original blog, and comments there should appear on Hacker News. (Except YouTube comments; no one needs to see those.)

I already mentioned a few of these points to the Disqus folks. They certainly have a good system on their hands, and any real competitor would have to do everything they do, and maybe add a few of the ideas here.
http://ikbot.blogspot.com/2007/10/server-side-video-processi...
http://news.ycombinator.com/item?id=65320
"Server-side video processing of life-casting robots"

I love iRobot's history of openness. The roomba API is completely open: anyone with a serial cable and some know-how can control it. The ConnectR should eventually be open as well.

I'd like to see this taken to the next level with live-casting from a robot. Justin.tv meets ConnectR.

The robot would either have to be managed by a human when in public, or it could wander alone in a private space. Community voting could control where the robot goes and whhere it directs its gaze. Ideally, the robot would be placed where average people can't be, like back stage at a concert or fashion show. Dangerous places are also ideal. Let the robot go where the humans shouldn't.

Robots with internet enabled cameras can do more than normal robots. Server-side processing means the robot doesn't need to have expensive hardware on board for the intelligence.

Obstacle avoidance, mapping, localization, face & pedestrian detection, object detection, object tracking & motion modeling, etc. can all be done using today's technology with a single camera stream. Some processing would need to be local given the constraints of network bandwidth, but plenty could be offloaded. It is also a good model for premium services: pay more to get faster connections, fast processing, and more capabilities.

Automated surveillance with alerts for intruders could be a killer-app for wifi/webcam robots. Today's choices for home security are immobile, cost thousands, or both.

Server side image processing itself is a viable idea. It's the next step in online video. Today, we just stream compressed pixels. Tomorrow, we'll calculate and stream information about the scene. You can send a flickr image stream to a service which finds faces, builds a corpus of data to identify the people from context, and performs face recognition. Face recognition companies like Animetrics could be tapped to do the hard part.

Motion detection for surveillance applications could all be online, with companies like Intellivid and ObjectVideo already having optimized the image processing component.

Street Views in maps can be combined with GPS tagged digital camera shots to build super high resolution aerial imagery, and eventually 3D.

Services like Fauxto which aim to be photo-shop online could build an interesting api where any image can be sent with instructions for processing.

The point is that all of these services require sending video and images to a server, where some intelligent processing occurs. Often times, the processing will involve the same software modules, so each problem is not unique. Tap the long tail of software development and allow 3rd parties to build their own processing streams that live on your servers. This could be made simple and standard using tools like Python with modules for image processing like OpenCV, PIL, NumPy, and SciPy.