skysight-inc/HackerNews huggingface dataset:
Just a quick comment. I had keepalive turned on for my webserver but set to a low timeout value of 2 seconds. The server is getting a lot of traffic this morning and was approaching maxclients. I had a few reports of pages or images not loading which suggested that the server was occasionally out of connections. I just turned off keepalive completely and wow! I've basically doubled (or more) the amount of traffic I can handle.

The down-side is that browsers need top open a new connection for every page component they load. But things will load more reliably now and I can handle more traffic.

I have a new server ordered which comes online today with lots more memory. When I deploy that I'll probably try turning it back on to make loading faster for my users. Then when things get hairy again I'll turn it off. Come to think of it it'd be nice if I could do that without rebooting apache based on server load.

Mark.
There are 5 other websites on the server. They're low traffic but some are PHP which means I've had to compile mod_php into the server so each thread takes up a lot more RAM.

I have a new server ordered which comes online today and that's just for feedjit, so I can compile a feedjit-only apache and it'll be able to handle many more connections.

Mark.
I've been thinking a bit about this. I have a couple of blogs of my own, so I always think about feedjit in that context. Am I happy with my readers seeing what search terms are sending people to my site and do I think my readers will get mad because people can see what they're searching on?

feedjit only shows the most recent 10 referrers and clicks, so I don't think theres anything there that'll give a 'competitor' some sort of strategic advantage. Besides, tehy can just google around and find out what I'm showing up for in the SERPS.

As far as privacy goes, as long as I'm not personally identifying people and showing what search terms they're using, I think there aren't any privacy issues. I see my own search terms showing up and my location 'Seattle, WA' and no one knows who actually searched that term.

I haven't really applied my mind to this as much as I should, but those are my initial thoughts and I'd love to hear if anyone feels different.

Mark.
(X)HTML/CSS Coding-Services
Anyone has data on the ROI of providing Open APIs? Are the services charging for use of their API making any profit? You guessed it, I need to justify the business case for Open APIs.
Sorry, I've been on the road - I'm busy driving from Denver to Seattle (currently in a hotel in Bozeman). I've replied to a bunch of questions and will check back again later today.

Mark.
Download "On Lisp"
c10k: Handling 10 thousand symultaneous connections.
A rewrite only tastes sweet after a long process of freaking out that you just threw away all your code.

And since time is the most precious resource in a startup then rewriting is the worst decision you could make, if that logic is correct.

However, Reddit isn't really a startup anymore, so they can probably afford it.
This is an oldie, but I haven't found any reference in news.yc so I thought it may be of use to some of you. Also, it's more up to date than it looks in the first few paragraphs.
One file per URL. The problem with MySQL is pretty much what you've described. I have (had) an index on a table that gets read a lot by the application. It's amazingly fast - MyISAM table's really rock for fast reads on indexes. But therein lies the problem because it also gets written to a lot. Every time it gets written to MySQL needs to lock the table and rebuild the index.

You can improve things a bit by using INSERT DELAYED. When you use that, mysql doesn't guarantee that it'll insert the row immediately, but the mysql query returns immediately when you do the insert (it doesn't block) and mysql queues up inserts and inserts them in bulk when it feels like it. The non-blocking and bulk inserts that INSERT DELAYED give you speed things up, but only to a point because you're still constantly rebuilding an index on a table that's getting a lot of reads.

Mark.
Now that's just awesome.
They're given to your closest competitors. Haven't you heard of Competitor Day? You'd better keep your idea to yourself.
I think at this stage both db's will probably do a fine job. PostgreSQL has been getting faster and mysql has more features. I just know mysql. I know where it keeps it's data files, how to recover from a crash, I know the my.cnf config file backwards and how to tune it, so it's really just down to what I'm comfortable with.

Mark.
Exclusive Download: Dine-O-Matic 2
definitely, they may (should) be looking at what they will need back-end wise for the next few years. if building on a dodgy base is too risky, you have to build the better foundation. they're not a start-up anymore, it's a new set of challenges
I thought this was going to be a thread about what people do to continue their StartUp after they are rejected. It could have been an inspirational story about striving through rejection and building something good.

Sigh
This seems to be like a site that is supposed to appeal to the general public. Aren't you worried that it won't get through the content filters at businesses and schools due to the name? Building your brand around cocaine seems pretty risky.

It seems like you feel that the API and the highly structured organization of the data is what differentiates you from Wikipedia. But, Wikipedia already has huge mindshare, and there are very few people that would ever your added value features. That means advertising is not a viable revenue source. Charging for access is out unless you change the name; who would you pay for something labeled "free"? So, what's the business model?

The people that WOULD need the data to be very structured need to have some assurance that the data is reliable. Usually, somebody who has spent a lot of time and effort to build an accurate data set will want a return on their investment. I could see somebody building a business by aggregating sets of reliable data from reputable data sources (universities, government laboratories, medical journals, etc.), and renting out the data sets to users and/or charging subscription fees. That is what junk mail companies due with mailing lists, for example. But, it is hard to imagine giving it all away for free and still making a living.
>I recall reading, though I can't remember where, that physicists in some country were more likely to become extreme religious fanatics.

i gave up at this point. or is he being sarcastic? i think he broke all the rules in the above paragraphs in that one sentence.
How do you brainstorm for ideas for products?
I recently found a co-founder and we are going to build a product. We are going to start brainstorming product ideas soon and I was wondering how the YC readers came up with their ideas.
Can you use something like this to change movie dimensions?

How would you incorporate the time dimension so that you do not remove parts of an image that will become important soon?

How would you improve the efficiency of the algorithm for a movie?
Alexis Ohanian: Early results from the redditor survey
What happened to you after you were rejected?
Has anyone applied to YCombinator, been rejected, then built something popular?

There was a line from the rejection email:

"We realize this process is fraught with error. It's practically certain that groups we rejected will go on to create successful startups. If you do, we'd appreciate it if you'd send us an email making fun of us; we want to learn from our mistakes."

Has anyone done that? If not, YCombinator picks the winners pretty well.
I agree. http://news.ycombinator.com/item?id=45619 :)
New Facebook Ads May Make Money, Upset Users
I don't think any of the applicants, accepted or rejected, are ready to rest on their laurels just yet.

I've gained users in the past few months, but am nowhere near popularity.
school vouchers don't need to be aggregated to the school level, just the teacher. a headmaster is then the facilities manager (school sports and stuff are a good part of school). you may not like vouchers and they are not the whole answer but connecting teachers to shock, horror gasp, the kids they teach has to part of the solution
A highlighted version so you can avoid the noise on first reading and go straight to the meat:

http://pastebin.com/f774fd6c1

I forgot to add before: if you have questions, I'm eager to reply them. If you have advice, I'm eager to hear it.
I never try to force it. All the good ideas I've had have just popped into my head when I wasn't trying to think of them, often in the shower or bath. However, this doesn't just happen without preparation. I do a few things to maintain especially fertile ground for new ideas.

First, I read a lot: the WSJ every day, sites like this one, articles on a variety of subjects, and books. Second, I free up lots of time to do nothing in particular in terms of thinking, such as going out to eat, taking long showers and baths, watching TV, etc. Third, I'm always looking for problems people are having. Fourth, I try talk to people about their general experiences on the Internet. Fifth, I bounce ideas off my wife. In your case, that would be with your co-founder. Finally, sometimes I just build things for a day or two on a whim to see where it goes.
I asked for one and they mailed it in a couple of days
I'm not sure if I'm missing something entirely but to what URL would you like us to bring our friends and earn these karma points? Might be helpful to get that out there.
Danny Hillis is associated with this company. They seem to be into the semantic web "thing" - not exactly a hot favourite with most programmers. Check out this podcast with one of the freebase guys - http://www.itconversations.com/shows/detail1879.html
The frontpage is extremely noisy. They didn't emphasize the correct things.
here's yahoo's version for yui: http://developer.yahoo.com/yui/grids/builder/
Great advice. Thanks!
Why you need a plan B in grad school
it's only depressing 'cause it's true
Hey palish do you have any screen shots on how the app is going to look like?
Stop Pretending you can Start a Company while Avoiding Sacrifice and Risk
Google announces a simple new way to embed Google Maps
Uberfact: the ultimate social verifier
I guess I'm just generally confused as to why AOL got in trouble. But IIRC each person had a unique idea so you could put their searches together, and I guess some of those people had searched for their own names. Since you're not putting multiple searches from the same person together it should be alright.

On the other hand, why should anyone expect their searches to be private at all? If I wanted to share searches coming to my website and even group them by IP address, what's stopping me? What if I said I would do this in my privacy policy?

BTW, I've been thinking of a Feedjit type thing but for search keywords instead of location. That's what got me wondering about these privacy issues. I hope that idea wasn't in your future plans ...
YC folks describe the accelerator effect of getting into the program. I think that makes sense. People expect non-failure from an openly declared intention to start a company. Founders respond to the fear of failure by working hard.

Many of those rejected probably don't consider themselves startups, just working on side projects. No build-up of expectation.

I have every intention to continue if not accepted into the next YC round.
Well in the case of my application, the submitter of that application, namely me, went on to live a meaningless existence.
Yep, I'd say that's a pretty good choice. Pylons+Mako+SQLAlchemy makes an excellent framework.
How To: Make a Laser Espionage Microphone
Britain enjoying 'digital boom'
That line you quoted from the rejection email is just perfect. It also shows quite the opposite attitude of the average big company. Reforming corporate America seems like a lost cause. It's time to start taking over.
Gambling Dispute With a Tiny Country Puts U.S. in a Bind
I'm very happy with the way things are going. I wouldn't call us a 'winner' (yet), but I don't think YCombinator's "rejection" is going to be the reason we fail, if we do.

Plenty of successful technology companies have been started outside of YCombinator. In retrospect, it's kind of a funny question.
Entrepreneurs need to see the full picture, all the time.

This is basically a stack of dualities:

Detail-oriented + Huge vision

Driven, focused + flexible, nimble

Enjoys relationships + makes tough business decisions

Relentless about customer service + Engineering-centric

et cetera

YOU HAVE TO HAVE IT ALL!
I second the doing nothing idea. I always sit down to watch Mythbusters, or Modern marvels, and well marvel at the time I'm wasting, but after the show is over I always have one or two new ideas. Granted not great ideas, but at least ideas.
Another idea is to keep a continual list of the all the problems you see in your life without automatically thinking of solutions right away. Then sit down later and see if you can solve any of them, or turn the problem on its head perhaps and then solve it.
If most people succeed because they openly declare their intentions, the reverse seems true too:

You're more likely to succeed if you openly declare your intentions to succeed.

And it's true. I'm not sure we'd be able to launch tomorrow if we hadn't posted our intention to. Now we have all your expectations to live up to, and it motivates us.
It should not have happened in the first place...
see http://www.paulgraham.com/ideas.html .

i'm not a PG fanboy, your question is just directly relevant to what he wrote and i think he's right.

basically, often times ideas that turn into billion-dollar ideas come from doing stuff smart people would already be doing anyway - writing BASIC interpreters (microsoft), good search engines (google), etc.
Crack?
But apparently we're still way behind Europe in terms of speed...
http://news.bbc.co.uk/1/hi/technology/6041446.stm

24Mbps is standard in France apparently, I need to get myself to Paris! :)
I have a friend in india that needs a visa. So I browsed around and found out about L-1. Basically, its a visa available to people working for multinationals. Legally, a multinational just has a corporation status in 2 countries (US and India in our case ). So I need to know if its possible to start a "paper" corp over there
True, lower the risk, lower the reward if it pays off. However, I think starting a business beside a job is a good way for people who want to start something small. Sure, they're not going to create the next Facebook part time in their garage. But they might build a nice lifestyle business that gives them quality of life.
That is deeply awesome. Only of use if you want to listen to a window that you can hit almost perfectly straight on though. They have the two components at quite an angle, that wouldn't be practical if you were trying to be discreet...
Well, first you get to rethink your idea and ask yourself" WTF is wrong here?" Then you start tweaking it here and there, making it simpler (but better) and a few months later you find that maybe the original idea wasn't all that, and the current one is more solid. Now you have been working hard to make the necessary changes and you are going to launch this Fall.
BTW the letter hurts.
This is a really tough one. Google are already taking serious heat over privacy and other concerns because of the amount of data they hold on users. Facebook will face the same challenges unless they allow users ways to opt-out, even if that means deleting your whole profile.

It's once they start actually selling the data though, that's when we need to really worry.
This is Arc, not CL. As part of the general plan to GC CL in Arc, I decided it would be enough to use one asterisk on globals. I now prefer one to two; an asterisk at the end is like one in text, warning you of an exceptional situation.
Top tip, it's a great table comparing the services side by side with links to their sites. I'll bookmark that one.
Silver surfers spend more time on the web than the young
What system do you use for bug tracking?
I am interested in seeing how you track bugs during development and acceptance testing.

We use bugzilla, but it leaves a little to be desired.
From my experience - likelihood of succeeding because you tell people and all the other variations of this are just hoopla.

You will succeed if

1. You have a market

2. You service that market better than most or better

3. You can do what you are doing for years at a length. And then keep doing it some more without even thinking about it.
There are also people in my position who have to start something big while moonlighting. I make 35k a year, so after car payments and living expenses (not credit card debt though, thankfully) there's simply no way to save up fast. I'm saving, sure, but at this rate it'd be a year before I could quit to spend 3-6 months on our startup. Godda start now.
I find it sort of weird to talk about these books as if they were O'Reilly titles or something; they're not meant to be "used" so much as studied (slowly!).
Sure, the idea has to be semi-solid. I was referring to the part of "Get this thing done, now. Get it out there and in front of people. Then we'll see if they bite."
I remember when I first did some web programming, way back in 1996, people were already warning against pixel-based code. Seriously, what happened? Did people decide it's just too difficult to do percentage-based layouts?
See also: http://brad.livejournal.com/2230083.html

epoll-driven jabberd from Brad "LiveJournal" F, scaling to 25k connections in 97Mbyte of RAM.
This is a great article not just about pitching to bloggers but sending press releases to anyone. The core of the post is a simple question, is your press release relevant? If not, save everyone's time and don't send it.
I enjoyed the talk, and I agree with some of what he says. My real problem with the talk is that isn't testable - it makes an observation, then goes back into the facts to justify it. I'm not seeing enough of the hypothesis-testing approach that is the cornerstone of real science.

I don't want to seem like I'm being too hard on the guy. After all, some questions just can't be approached this way. We can measure the effectiveness of a new drug on blood pressure through hypothesis and testing, but it would be very difficult, maybe impossible, to use this approach to determine why the first world war was fought. Sometimes you just have to go back through the facts and construct a plausable case. But it isn't science anymore.

Aside from that, I agree (for what it's worth) with much of what he has to say, but I still think he has understated the degree of exclusion women have experienced from male networks. It's really damaging.

Suppose that it's true, as the author stated, that men are more likely than women to be wired for risk. Sounds reasonable enough. But that might still mean that for every 5 people with the ability and inclination do do something risky (say, found a startup), 4 are male and one is female. But the way males form networks often excludes the women - even in the modern day. I don't think it's on purpose, and there's rarely any malice, but it's just as damaging.

You generally pick friends for your startup. The people you get drunk with, stay up all night completing the compiler class with, cram into a hotel room in vegas with... and you tend to do these things with close friends who are almost always other guys. If 4/5 are guys, then the odds that a man will find like-minded buddies is much higher than it is for women.

I double majored in Math and English. It was a trip. I'd count the ratio of men to women in each class, and it was usuall between 5:1 and 10:1. And in the math class (sorry to say it), many of the women were planning on becoming math teachers.

This means that the women with the ability and inclination to do startups have to break into the boys club or they'll be left in the cold. So far, they're usually left in the cold. If this weren't the case, I have no doubt we'd see far more women in fields that require a high degree of risk, creativity, and collaboration.
Facebook Finally Got Interesting
Brainstorming is a waste of time.
Facebook proves its banners have no value worth mentioning
Check out Roundup - http://roundup.sourceforge.net/ .
I'm always confused by people who don't have an idea but want to start a business. Surely, an idea has to come first (even if the business ends up doing something else).

I maintain an ideas list which is added too weekly. They're not all gold dust and so far none of them have been strong enough for my to quit my day job but I am interested and excited by a few and actively working on them. I think you need to have a desire to build XXXX before you have a desire to build a business, otherwise you're putting the cart before the horse.

Anyway, with that useless rant out of the way, the best way is simply to think of something that annoys you (or annoys someone you know). Actually, don't think of the problem, think of the solution when you next come across an annoyance. And don''t go for something just because it sounds "cool". There's lots of talk about finding the "pain" that a particular customer/market has and solving it -- I've heard it so often it sounds corny but it's not.

Here's an idea that's been sitting on my list for a while: new album notifications. I don't subscribe to individual band newsletters or read music magazines, and my favourite artists aren't necessarily heard on the radio, so how am I to hear about their new releases? Well, Windows Media Player has a comprehensive list of my music and what I actually listen to (an a plugin interface). How hard would it be to poll Amazon once a month for new albums from my 20 favourite artists? Business model: Amazon affiliate sales by allowing users to buy/preorder through your application. Sure its not the best idea in the world but just an example of how I went from a problem to a solution (aka a business idea -- and, no, I'm not working on it).
sell your car, move with your girlfriend/boyfriend and code till you get a call from Yahoo.
Anywhere.fm When is the API coming?
Web Science: A Provocative Invitation to Computer Science
That's.. certainly one way to do it. I'm debating that, but it's another thing when the chance of you failing affects someone else (for example, if you have a family). Technically you can't 'fail' at a startup, you just quit working on it, so how long do I invade their life? I couldn't bear to ask Emily's father to support me for a year or more.. It just feels morally wrong, especially when he's already 20 years in home debt and is supporting three children and a wife.

Does it make me a non-entrepreneur to worry about morally wrong things? I don't know.
Ick. I would never rely on so crude a tool as Notepad for my bug tracking. I use emacs (text-mode).
http://16bugs.com/
We got rejected, then started a site that got Techcrunched a few months later (thought it didn't end up taking off). Now we're working on another company that is funded and has a growing user base on Facebook.
Just curious, how'd you attract TechCrunch's attention?
Trac ( http://trac.edgewall.org/ ) is awesome. It also has very good integration with subversion, if you want that (but you can ignore that aspect if you just want a bug-tracker).
Kevin O'Conner, the founder of DoubleClick, answers your question very well in his book: "The Map of Innovation: Creating Something Out of Nothing". It's an excellent small book that you can read in an afternoon. He explains his brainstorming process and puts a lot of emphasis on selecting the right idea before spending too much time and effort building it.
Comments at the top of the file.
I fix bugs before I need to track them.
On personal projects I use comments like

(defun foo (x y z)
;; FIXME: foo should not barf if x is nil

and then periodically grep all my sources for "FIXME".
Going for full Notepad(tm) compatibility, eh?
I surrender. Lisp really is better than C++.