skysight-inc/HackerNews huggingface dataset:
40 and I still feel like a programming noob, but I've only been at it for about 12 years. One advantage I think those of us who work for big corp may have is that we can see inefficiencies all around us and inefficiency = opportunity. May not be as sexy as the latest Ning, reddit, Facebook, twitter thing though.
I've heard OCAML rocks for image processing. This page is really cool.
Yes, most of true SEO (read: not blackhat tricks that will get you burned in a few months) are just standard practice.

Like starting a blog thats heavily linked to about your topic of expertise.
4 co-founders... who's on the board and how would you determine that ?
I have been told from potential investors that having a large board could be a potential hindrance to further investment. Any advice on this issue...has anyone gone through a similar situation?
I'm 39, and have been developing my site for about 4 years:

http://ourdoings.com/
This Day Will Be Remembered: Facebook, Google And Plaxo Join The DataPortability Workgroup
Official Google Blog: The flow of information at the Googleplex
I'm 53, almost 54. Not everyone over 30 is a technophobe or afraid of change.

Access to computers was very hard to come by when I was in my teens, but a lot of folks were interested nonetheless. By my early 20s the first PCs were hitting the market. A 48K basic Apple II cost about $2000, equivalent to perhaps $6000-$8000 today. As poor as I was then, I bought an Apple II system in 1979 and have maintained a strong interest in computers and technology ever since. People in my generation (and slightly older) started the personal computer revolution, and much of the earlier work on the internet.

One advantage of age is a perspective at how far we've come. In the early 80s I was a systems programmer for a large IBM/370 shop, running IBM's VM Operating System... the original "Virtual Machine" environment. We supported 500-1000 active/logged-in users on mainframes that had 16MB of RAM and perhaps 8GB of total disk storage. It was pretty cool in its day. Today, a $400 laptop could whup that system (in most respects). Amazing really.

My colleagues and I have gone from programming in PL/I and Assembler to Basic => Modula2 => C +. C++ => Java => Python/Ruby => and lately, a growing interest in that old warhorse, Lisp.

I can't sprint as fast as I once did, and it's harder to engage in marathon all-nighters than it was in my 20s, but in most ways I'm smarter and more capable now than I was back then. So please excuse me (and others like me) if we continue to stay engaged...
So has "sneak peak" just evolved to post-ironic usage now or what?

"Sneak peak" is more popular now than "looser" ever was.

http://news.google.com/news?q=%22sneak+peak%22
Guerrilla Marketing + Lost and Found = Light Bulb (good startup marketing idea)
MS Bids $1.2B To Buy Norwegian Search Company
Also not sure if I would hire him as a comet developer...

(OK, bad joke from a response in another thread)
No established site is going to want to use that for its authentication system. They don't want to make it easy for you to log into sites other than their own.
Farmer Manages Cows with GPS Device
Article shows the extent that things are digitized but argues that there is so much more to come.
Any chance you can make it do ssh? That would be awesome.
Consideration 1: How many readers will go away if monetization feature X is added?
Consideration 2: How much money will feature X make?
Consideration 3: Does feature X fit in well with the rest of the blog?
You can find a reason to discount anyone... chemistry matters too
Consider two things: 1. interview questions may be more to do with finding out how people answer, 2. posting (possibly sensitive & flawed) recollections of an interview reflects a lack of acumen.
I was about to submit this one myself...

The article referenced does identify a major problem in an undergraduate programming curriculum. I like the idea of a BFA in programming (this idea has been around for a while), as well as a rigorous program that requires a much deeper understanding of programming languages. Java schools are a problem, no question about it.

But I'm a little sceptical about how much it will help in the grand scheme of things. I'm starting to figure that trying to create a truly talented programmer through a CS curriculum is like trying to create a truly talented novelist through a creative writing program. An intellectual environment is valuable... but how many students are there who would have learned the things listed in this report (functional programming, proper memory management in C, LISP, ada, proper use of threads, a deep understanding of what happens under the hood, as well as solid mathematics) if and only if the curriculum had been different? Or would these students, who will be few under any circumstances, find their way to these subjects anyway?

I don't know the answer to this question. I have a feeling that a better curriculum is worthwhile, and would improve the number of excellent graduates - in other words, while I think that some will find their way to this material on their own, I do agree that there are people who will not achieve the talent level they are capable of because of misguided course requirements. But I wouldn't expect this to turn lots of people into highly talented programmers any more than a writing program is going to produce a bunch of spectacular novels. It just ain't in a lot of people (including, I'm starting to suspect, myself).
Because it brings no benefit to anyone. Not to the one who runs the site and not to the one who signs up to it.
Presumably the people reading your blog have similar interests or circumstances to you. So you could ask yourself what kind of ads or other stuff you'd want to see if you were the reader.

YCombinator seems to be a good example of monetizing what is more or less a <a href="http;//paulgraham.com">blog</a>. I guess the generalization would be to start a business that you and your readers would like to use.

Other than that, I think there's a perception that web ads are sleazy and that clicking them will lead you to someone who is trying to rip you off. The ad either needs to be from a known company, look like it was made by real humans instead of a corporation, or be from sponsors you know the site owner has dealt with personally be cause s/he sells hir own ads. Come to think of it, maybe some of your readers have things they want to advertise.
On the ever expanding TODO list ;)
Looked into it and it's not too hard to incorporate.
I agree with everything here. If I had to pick out the most important, it would be the last sentence of the second (and seventh) item: "Now you've got to build something great."

Few startups build something great and then die. Nearly all startups that fail to build something great die. Therefore succeeding is identical with building something great, plus or minus about 1%. In phase 1, doing anything else except making something great (fussing over paperwork, arguing with one another, working on other projects, worrying about competitors) is a mistake.
Hacker News Documentation
Is there something like this?

I just noticed the "saved: 7" in my profile. And my guess was that this is a list of Comments I upvoted. I tried by upvoting a random Comment. And it seems to work. But how do I get rid of it again? Or in other words - how do I downvote? Or more generally: How do I find out how to do stuff?
32, and loving the startup life.
How do I find out how to do stuff?

The same way you do on an iPod: try things.
So that makes this guerrilla guerrilla marketing?
This is a really great essay. I go back and read it every year or so. There is actually an 'updated' version, not done by Stephenson but with his blessing. It takes into account the rise of OS X (which Stephenson loves) and updates some of the other arguments. Here is the updated text http://garote.bdmonkeys.net/commandline/index.html
Yet another technology looking for a market application...
Arc is a language implementation, not an application.

Optimization of language implementations is arguably never premature. When such an optimization translates to an application speedup, it's indisputably not premature.
http://searchyc.com/comet

FWIW, it's gotten quite a few mentions here at hacker news.
Sure, but what if publishing to the web is about ~10% of your business, and not your competitive advantage?

Do you really want your primary developers coding up a new framework?
Sounds like Microsoft joining the web standards committee, at least in Facebook's case. They have no incentive whatsoever to make the data on their servers portable.
"I'm starting to figure that trying to create a truly talented programmer through a CS curriculum is like trying to create a truly talented novelist through a creative writing program."

On news.yc we tend to only talk about truly great programmers because that is what is required for startups, but I feel like there is still some role in the world for programmers who are merely competent and good at what they do, in the same way that there is still a need for proficient writers to produce technical manuals and ad copy.
Hi. I'm going to stick this sticker on everything I see. You, by default, will see the stickers. I will now call myself a guerrilla marketing guru. Pay me on the way out, but not before I stick this sticker on you.
Kevin Rose supports Ron Paul, Barack Obama
Some people may argue that this is why Digg's front page is dominated by Ron Paul stories. On the other hand, it is quite refreshing to see a media mogul openly declare his political preferences. If Rose has any bias at all, at least he is completely open about it.
It's already used in applications.

IMHO It doesn't really need a 'buzzword'. It's just a common technique.
Guerrilla marketing is an unconventional way of promoting on a very low budget, by relying on time, energy and imagination instead of big marketing budgets.

I would say that the example is guerrilla marketing.
Rich Skrenta: Another way to look at Wikia Search (it's more promising than you may first think)
On news.yc we tend to only talk about truly great programmers because that is what is required for startups

While truly great programmers will help the business become profitable and successful sooner, they are by no means required.

I think there are plenty of non-vc funded startup businesses out there that do ok with so-so programmers. You just don't hear about them in the media because they are privately held.
Ask YC: Any ideas about intelligent crawlers :)
Hi everyone. Not sure if I should be posting this to a forum or not. Curious to see what peoples answers are here as I read this site a lot

I'm thinking of creating an intelligent crawler in Python. I have a project with a friend where we'd like to crawl a few specific car-related websites, grab some of the info and look for new entries. I am wondering if there is any existing technology out there where a crawler is sent to a site and either trained (visually?) or which can understand repeating information like tables that we could use to create a proof of concept. I'd appreciate any critique of my ideas which is:

1. create a visual tool - probably windows/mac based which uses the browser to navigate a site and to highlight elements that we would like to capture, such as car name, description, price. This would also have to be able to automatically/manually work out repeating elements

2. this tool would create some kind of file (xml?) which would then be used by the main crawler to understand how to navigate the site

3. The crawler, which we'd write in python would visit the site every week to look for new information

Am I going about this the right way or does anyone have any ideas

One point, we would seek permission from the sites before crawling - it would be to their benefit as we're looking to push people their way.

Appreciate any thoughts anyone might have

All the best

John
So, you're basically asking for comments? (as in code comments, not snarky responses) ;)

Some things are tied to karma, having karma above a threshold value will allow you to make a bigger mark on the site.
I was disappointed by the lack of actual Blub. This is News.YC after all. Sorry to be a typo totalitarian.
The CEO is typically on the board, and/or the most senior and experienced technical (co)founder in most cases, as far as tech startups are concerned.

Rarely have I seen or heard of more than 2 people in total from the actual company itself on the board for at least the first several years.

Also, from my observations, the number of founders on the board is probably roughly proportional to the amount raised, and the perceived value of the founders. IE: it wouldn't surprise me to see a startup that raised $80MM in capital to create a new hybrid commuter vehicle, with GPS-based auto pilot, to have 3 founders on the board. As long as those founders also have previous experience/wins to justify their board seats vs. just having a really good idea.
8 steps for fixing other people's code
What you are talking about is the deep web I think and I don't think anyone has managed this yet! Essentially you want a system that can fill in forms and pull back results. I think it needs to be done on a per-site basis
If nothing else, publicly speculating that your interviewer is bad at his job, is somewhat less than tactful.
Shorter is better (- or should I say cynical?):
"Not sure if I would hire him as a Java Developer, though."
Yeah, I thought this was going to be an article about creating a lightweight language...
CES Special: A Chat with Microsoft's Bill Gates
>There are already several programs going in this direction: a lot of Canadian universities, notably Waterloo, have Software Engineering programs...

Waterloo has Software Engineering, but its not really aimed in the direction he is suggesting. Software engineering is basically Computer Science (math dept) and Computer Engineering (engineering dept) combined. Really, I do mean combined, because it approaches double the work of either program, or it did a few years ago.

What Waterloo has done to address the issues Joel presents is to create a new Bachelor of Computer Science (not to be confused with a BS, it is a BCS). This program is designed to reduce some of the math requirements in later years and allow individuals to have more upper year CS courses. They have not eliminated any of the math or theory out of the curriculum in the first few years.

Waterloo has not had a problem teaching Java to first year students, mainly because the focus is not put on the language. They don't really have any courses designed to just teach you a language, the goal is to teach programming concepts. Early on OOP, data structures, and so on are taught, then things move into algorithms.
Which trend is for you in '08?
I use the "disposable" card numbers whenever possible, and get a new number issued about every 15-18 months on my primary credit card.

I know I bought some stuff from them, but I forget how long ago it was. Pretty sure it was before my most recent numbers were issued though.
"The teachers are experienced software developers from industry. The studio operates like a software company."

Oh, you mean like ycombinator?
Don't write your own crawler. Use nutch.

It is designed to scale and do mapreduce kind of parallel processing. I would strongly recommend you to take a look before writing your own.

http://lucene.apache.org/nutch/
Yeah, but they're marketing the product by marketing the guerrilla marketing. That makes it guerrilla guerrilla marketing.
:) My goal is to make a post like that every month.
I like how you styled it as advice for Y C founders, even though it's applicable to all startups, hoping that our competitors will therefore ignore and/or do the opposite. Brilliant maneuver sir.

Remember if you're not a Y C founder, be sure to spend all of your time working on patents.
How did they come up with these numbers?
I would really like to see a real name in each category ;)
we tend to only talk about truly great programmers because that is what is required for startups

That's not what we look for. The evidence so far (and there is now quite a lot of it) suggests that it's more important that the founders be very effective people than great programmers.

I suspect this is true even in research, but certainly in startups, someone who is reasonably smart and very driven will beat someone who is brilliant but ineffectual.
Radical constraints can lead to breakthroughs in simplicity and entirely new things
The Truth and the Hype of Hypnosis
What's it doing is abusing a useful social system where lost articles are returned to owners. It's not guerilla marketing, it's spamming.
"This page originally continued with an anti-Microsoft jeremiad. On reflection, however, I think I'd prefer to finish by thanking the principal authors, Vinod Valloppillil and Josh Cohen, for authoring such remarkable and effective testimonials to the excellence of Linux and open-source software in general. I suspect that historians may someday regard the Halloween memoranda as your finest hour, and the Internet community certainly owes you a vote of thanks."
You've answered your own question. One of the (many) advantages of releasing quickly is that it's better for morale.
I see your point. True.
It's been pointed out to me that "Fail" is hardly constructive criticism. I'll try to update my blog post (linked below) with something that might actually help.
Show me any example of "guerrilla marketing" that doesn't take advantage of some system. That's what makes it "guerrilla". This example is no less guerrilla than SPAM itself.
And still one thing .. if you want to ask the site owner for permissions, why not ask them to produce some specific xml file for you?
Because granting permission is easy. Why would they go to more effort than that for random people?
Trans just sent me a link to this thread... is that guerrilla guerrilla guerrilla marketing?
What do you want to do?

At least in Germany, there exists a few solutions which do exactly that.
If a person puts his car on sale (a bargain) on one of the car related websites, he get's the first call in about 20 seconds from someone using these programs.
My college switched from C++ to Java based coding in 2005 for learning the fundamentals. I would have much rather learned C++.
Why thousands of job sites produce custom xml output for simplyhired or indeed ??
I think you are in over your head, but it's a great way to learn about the plumbing and underbelly of the Web.

This visual tool is basically what a company called onDisplay was doing back in 1999, before they were bought by consulting firm Vignette for an obscene amount of money. But scraping against the html structure is a losing battle.

A better approach is to use clues in the information itself to guess its content: something with a "$" is a price. Something containing "toyota" is probably a name, "blue" a color, more than 20 words containing "good", "v8" is a description, etc. That way your scraper is resistant to structure changes.

All that is separate from the problem of a crawler. It takes a long time and a lot of effort to convince content sites that what you are doing is a) helpful to them and b) something they should not be doing themselves.

It's like jumping on stage with the band and starting to play. You better be really good and friendly and prepared to get the crap beaten out of you.
I remember seeing a screencast of some startup that did something like this. It was maybe a year ago. You click on elements, it shows you other hypotheses, you correct if necessary, and you get an RSS feed compiled from the page structure.

Anybody remember this?
Some fruit doesn't hang quite that low, though.
I know in php it's possible to load an html document and parse the DOM tree using XPath expressions, presumably that capability exists in python.

So i guess in theory you could write a frontend (firefox extension?) where you could highlight / select a screen area (webdeveloper already does this), then pass it's DOM information (i.e. #body table tr td#username ) to your backend, which would then scrape that field(s) from any applicable site pages.

This of course assumes that 1) The website(s) are well formed enough for your parser and 2) Well programmed enough that the same info is in the same place in the DOM tree, and preferably ID'd, which are pretty HUGE assumptions, but could be worked around if you were determined enough.

Not sure if this is what you're looking for, and it seems a bit circuitous, but it's a plausible idea anyway.
Way back during frosh week in university, I went on stage during a hypnosis guy just for fun.

I tried to get into it as hard as I could and go with the flow, it wasn't happening. My friends on stage who did get 'hypnotized' said they were just doing whatever he said to act foolish and have fun.

Either way, it turns into an entertaining stage act having some people running around doing whatever you say. Convincing frosh to do stupid things is as easy as depressing teenagers.
I have built up something similar to what you are describing and it was a fun project. My first reaction to #1 is that if the information you want to repeatably in the same place you are probably better off just doing things manually and not going to the trouble of building a visual tool.
In my experience the interesting pieces of data tend to move around and something like regex is the best way to handle this.
I used wget to grab data because it was quick and easy. I then did the post processing in the background, separating out the grabbing of data and the interpretation of data.
For 1 you mean you want to build a parser for arbitrary html that your crawler returns. Hard problem, as others have said. My advice:

1. Use an html parsing library. Beautiful soup (python) or hpricot (ruby) are good building blocks.

2. Practice manually building parsers for a few sites, then see if it leads you to any insights about how to generalize the process.

3. Ignore everything else until you do 2. Just use wget as your crawler. Skip the visual interface for now; just parsing arbitrary pages is a hard enough problem to bite off.
VideoTrace: Very Impressive Project for 3d Modeling from Video
Microsoft Bids for Norwegian Search Engine Firm
Tutorial on embedding concurrent Erlang into Ruby
Another talk I saw at shdh in october (http://superhappydevhouse.org/SuperHappyDevHouse20):

http://tagtheplanet.net

They seem to be attempting an intelligent crawler as well.
I agree that this is the potentially "hottest" statement from the story. Yet only two examples are given: Google and Twitter. Taking away the clutter around the search box was a good idea, but wasn't Google's real innovation something like counting links to pages to judge their significance? The Twitter example is better, but one example is not enough to make the case. Are there others?
That's precisely what I was thinking. Until I see an actual implementation, I'm assuming this is just a token gesture.
Nice quote. According to Google it's on the net in only one other place:
http://theaegispress.com/Philosophy%20of%20Goethe.pdf

Which turns out to be a pretty good article. The phrase after this quote is pretty interesting too as a summary of what is desirable:

"Always striving and always working toward a new synthesis.."

Read the whole thing.
Evolutionary Theology: How to Love God and Science | Wired Science from Wired.com
You ought to check out screen-scraper (http://www.screen-scraper.com/). It's a commercial app, but the best I've used for this kind of thing. They also offer a freeware version.
dapper.net
Evan gave a talk on this at Le Web 3.0. It was short and sweet. I liked it, it makes the point well. Found the video online: http://light.vpod.tv/?s=0.0.392057