skysight-inc/HackerNews huggingface dataset:
Yes, you'd have a better chance of accurately estimating the risk, and you would be in a position (control) to do something about it, trim the sails, fire the management, retain the management, whatever. Staying involved day to day has costs for sure, but Carnegie and others in the 19th Century seemed to model a way where they spent a half day on weekdays at best managing many, many businesses. It probably has a lot to do with whether you can motivate and retain good management to work for you.
Ruby is also a better engineered language than PHP in a lot of ways.

I would say better designed, but the engineering definitely leaves a lot to be desired. Here's hoping Rubinius gets finished one day.
Yes, you have to build the shelf life of the company into this. One model I have seen a Seattle investor use is to assume a shelf life of 12-15 years at best. But if you can return invested capital every 2-4 years, you have a good run. And if someone offers to take you out, you can measure your opportunity cost against the offer, i.e. you can determine whether the buyout makes more sense than running the company through its natural life cycle.
Ruby wasn't even a semi-mainstream competitor for web development until Ruby on Rails came along, so even comparing it to PHP (or worse, thinking Ruby and RoR is the same thing) is where a lot of the confusion starts.
Yes! You could easily sell users a service that helped them hunt down and delete their old embarrassing geocities sites, if you could figure out a way to confirm that they are indeed the authors.

People could use google, but you could put extra work into brewing up some special sauce that would, for example let them find their sites with only a combination vague memories, such as their neighborhoods, or when they created it, or the types of things they linked to, or the background music their site had (mine had the mission impossible song) or the type of content (mine had lots of animated gifs). Google wouldn't care enough to do that.

If users find their content on their own, they can always request it to be removed, all you're selling them is a tool to help them do it.

I wonder if people would really pay for it? It would be fun to find out. I would help build it
Amazon EC2 High-Memory Instances (34 GB, 68 GB)
Ask HN: Building a domain-specific search engine. Advice, experiences?
I've been thinking for a while of building a domain-specific / "vertical" search engine. I usually end up telling myself it's crazy to think that I can compete with Google, but the recent thread (http://news.ycombinator.com/item?id=902999) on the apparently declining quality of Google search, especially for niche queries, got me thinking that it might not be so crazy after all. What's more, I know of at least one existing search startup in this domain who seem to be doing well, so that's some additional validation of this market.

Although I'm a pretty competent hacker, I'm new to search. I've done some reading on search and information retrieval in general, and picked up a copy of Manning et al.'s "Introduction to Information Retrieval", (http://nlp.stanford.edu/IR-book/information-retrieval-book.html) which looks fantastic.

However, I don't want to reinvent too many wheels, at least not until I really have to. I'm planning to dive into Hadoop, Lucene, Nutch, Solr, and other open source tools and see how far those can take me.

I would appreciate any advice on this endeavor -- preferably from your own real world experience -- tips, tricks, pitfalls, resources, etc.

One particular issue that I've pondered is how to do a good "targeted" web crawl -- how to restrict the crawl (or at least indexing) to pages I know are relevant to my domain. It occurred to me to seed the crawler with a set of "authoritative" domains that I know are relevant, then use pages from those domains to train a simple classifier to apply to each page visited, to decide whether to index it and crawl its outgoing links. Any other (simpler?) strategies?

Also, how far can I realistically expect to get with "off-the-shelf" OSS tools like Nutch, etc.? Those of you who've used such tools "in anger", what roadblocks and brick walls can I expect to run into?

Thanks in advance!
15% price cut in EC2 instances, effective Nov 1
Oh, now i understand. Then distributing it would still work then? (or would have worked i guess). Sorry to belabor that point, I'm just learning about this stuff, and I want to make sure I understand it, and you seem like you might be able to answer the question :)
Amazon constantly impresses me with their AWS offerings and improvements. It's really refreshing to see them continue to build and improve their services so rapidly. Sometimes it seems there is a new feature or announcement every week.

Now they just need to open that west coast datacenter. :)
What you are saying is that massacres are easier with guns.

Killing one person with a knife is not harder than with a gun. A criminal willing to kill is going to do it with one, or the other, or both.

Just look at the UK. Violence is bad and there is a total ban on handguns and a near total ban on just about everything else (many restrictions and you need a purpose to own the gun - self defense and for "fun" are not reasons - eg. farming, deer huting, formal target shooting )
R 2.10.0 is released
For those interested, here is a link to the JT API:
http://apiwiki.justin.tv/mediawiki/index.php/Main_Page
This piece is a bit much, but I do think it's odd for Joel to be prejudiced in only hiring people from top schools and then complain about how top schools are too theoretical. You can't have it both ways.
"I did notice that PHP developers favor SVN to git..."

I can attest to this. I spend a lot of time working with PHP with a lot of different people, and almost every single one of them uses SVN for deployment. I'm not sure if this good practice or not, but it's just so convenient.
While I mostly agree with your point, the phrase "Infinitely harder" is a bit of a stretch. I've worked with many professional software engineers without CS degrees, and even many professional software engineers without 4-year degrees at all.

Personally, I was already a programmer when I took my first University CS courses and realized quickly that while I love programming, CS isn't where my interests lie. I took the CS courses that were most relevant and wound up getting a business degree.

I do wish there were more programs available serving the middle ground between the v-school ghetto and the ivory tower.
Garmin Comparison Chart
Nice comparison chart with the amazon.com's prices are showing.
Yes, absolutely. That's how most of the work got done. The biggest problem when you start distributing it is to avoid duplication.

I took some shortcuts there, so I'm fairly sure that a portion of what I've downloaded is in duplicate, but that will be resolved in a merge step.

Right now the files are spread out over 7 machines, the one I started on is the 'master', and then there are 6 others that have a portion of the data on them.

Each of those has been told to fetch only from a restricted area of geocities, but the master one had no such restrictions, so chances are there is some duplication between the master and the individual slaves.

Merging all the data and importing the user accounts is going to take a couple of days at least, it's quite a collection of files. I have no stats yet but when I'm done I'll do a write-up on the main statistics.
It would be great if a award, with a substantial prize (upwards of $10 million), was created to be awarded to the person that has done the most for the human race: for peace, heath care, human dignity, human rights and freedom.

Bill Gates .. you listening? ;)
Google Voice with your existing number
$62 per month for 1.7GB is pretty good. For comparison Slicehost charges $70 per month for 1GB. I just wish they had smaller instances.
A recent Gillmor Gang got pretty hot over it.
No, though if it's an alternative idea you're really serious about you could file a second application for it under one of your cofounders' accounts.
And there's no reason that those who are drawn to the purity of math and science behind computers can't attend traditional CS programs. It would be good if real universities could also (as a different department, whatever) provide relevant instruction and research in the realm of programming.

Those who really care about both can always double major.
Question: Does the sidewiki area indicate in any way that it is not part of the website? Does the url bar only show the sites url, or does it also show some indicator of the side wiki?
once you know how to use the tools, it is good to learn some theory... for layouts check out Grid Systems in Graphic Design, it gives you a very Swiss approach for building a grid and then fitting content to it.

Khoi Vinh is another guy to look up, he heads up design for nytimes.com and is a huge proponent of using grids his site is www.subtraction.com

Also, learn about typography. Elements of Typographic Style is supposed to be best.
Closures, namespaces, objects, mixins, meta-programming, etc.

Yes I realize PHP has made strides in some of those areas, but its mostly too little too late.
Yeah, I'm a little US-centric because I know the local stats better. For the most part, I blame relative economic class and family problems (e.g. abuse). I find blaming an object to mask the real problems in our lives. It's like blaming the engine when you run out of gas.
I was not a Obama supporter but I really did hope that he would bring transparency into government, which is badly needed. Doing so would have used up a lot of his political capital. It would have been a great achievement and I would have cheered him on - regardless of what I thought of his policies.

I am saddened that he did not follow through and it seems unlikely that he will in the future.
Ask HN: What is the value behind OpenSocial?
After reading through Opensocial's site and documentation, I find myself still confused. Perhaps part of it centers on the fact that I wasn't around when it launched.

Is it a failed attempt at making apps adopt a universal api? And I say failed because Twitter and Facebook aren't part of the movement. Or is it going to be a force to be reckoned with in the future? Also, how does it fit into the Wave's upcoming launch?
$1.20/hr @ 34 GB, $2.40/hr @ 68 GB
Today's high end handsets are tomorrows cheapo ones.
did any particular tutorial make it click for you?
You can now query likes & shares for any URL posted on Facebook
I thought it was excellent. I have this kind of anxiety problem all the time, especially when writing new code, because the code I write might be wrong, so I end up just sitting there, not solving the problem, not thinking about the problem, but just staring.
Except that on Slicehost you get 400GB of transfer which would cost $68 more on EC2 (outbound).
What an incredibly lazy, link-bait, uninformative, unoriginal headline.

It's about the Google Sidewiki - http://www.google.com/sidewiki/ - released just over a month ago.
From a practical POV I don't think you could ever, ever, ever get around the perceived privacy issue of using cell phones. (Maybe even a real concern.)

As to the using the bus data I have run some tests using the SF Muni dataset (nextbus.com). The speed of the buses in the dense parts of the city doesn't correlate with the speed of traffic. They stop too often and the timing of the stops is inconsistent. Here in SF the buses often move over a fixed distance (20 blocks, Van Ness) in the same amount of time at 2AM and 5PM.

Dash is a great idea (we own two!) but obviously the money hasn't worked out.
Facebook might be bigger, but I wouldn't call OpenSocial a failure since around a hundred million people are using it.

I've been writing an app that is supposed to work both on MySpace and Mixi.jp (~20M users), both use OpenSocial. While theoretically you are supposed to be able to write once and run everywhere, in reality you have to tweak your app to handle differences in feed publishing details, checking call signatures etc.

Still it was a lot nicer than having to write the app totally from scratch for each.
I really wish Google had the option to blacklist certain sites from the results, such as EE. Maybe they could even use the data of what people are blacklisting.
Rackspace's version of Slicehost has 1GB for $44/mo and 2GB for $88/mo which is in line with EC2 but more granular.
Imagine if Google actually did something evil? We'd have to invent a new term for it. This feature could be annoying, sure, but if you're the type of person who installs Google Toolbar that's probably not a big concern. There are many times I would like to be able to publicly flag a site for inaccuracies, outdated information, content theft, or simply say I think this guy is silly and you shouldn't listen to him. I understand why the site would want total control but it's the Internet -- it doesn't work that way. The end user, more or less, has total control over how they want to get information.
Christianity != Creationism

Agnostic != Evolution

Christianity != not(Evolution)

Most Christians (globally) care little about how man came into being and do not think either ideas have any impact on their faith.

It is the hardcore creationists who like to make out all Christians agree with them, and the hardcore underdog atheists who like to make out that all Christians are against them.

I don't know any Christians, and I know many, who believe that evolution and the bible are mutually exclusive ideas. They simply are not. And I wish both Atheists and Creationists would stop claiming that both ideas contradict each other.
They are cosmetic....check back in a year and see if the money transfer moved to a new channel or not.
I wonder how:

http://fathomdb.com/about/home

will respond.
Whether it's cheap or not is dependent on the amount of data you transfer, at $.10 / G they will charge you $33 / month for a continuous 1Mbit link (328 G of data traffic).

If you're a smaller user then the price is much higher still.

For comparison, buy bandwidth in bulk and you can expect to end somewhere between $3 and $5.

As long as those bandwidth prices don't come down I can't make the figures work for web stuff, unless it is something that periodically needs a large number of machines. That's when it makes sense.
You should really rethink your job as a railroad engineer.
Maybe I'm just ethnocentric, but I don't like all the foreign (greek?) text. It confuses me. When I click the english flag, it doesn't make the greek go away.
Yeah, I know that. I had about 40 tabs open with every page I could find. All I'm saying is that It takes a while to figure out stuff like:

I would like to call toLowerCase on a list of strings. How do I pass an argument to the function?

(map #(.toLowerCase %) word-list)

It's not exactly obvious without a bit of searching what # and % and . mean here.
None of those features are necessary (or even desirable) for building large scale software. Very few languages have that full set including C++, C, and Java.
I think most people agree that universities should try to teach people how to learn, and not just the latest technologies and good engineering practices. But I've sat through too many university lectures on programming language syntax or introducing threads to believe the university is teaching me "how to learn".

Why not tell students, "here's an introduction to C++, read it before next week's lecture" or "for this project you must use a VCS, pick one and learn it" instead of making me sit through lectures where the professor reads documentation to me? Have other people had the same experience?
Correct. Guns are not the problem, culture is the problem. But if you have a bad culture, firearms can make it worse. But the reverse can also be true.
Use Google Voice with your existing number
I'd read the guides and then try to understand how a simple example application works. Then, try to deploy it to Heroku, make some modifications, and see your changes in the real world.

Here's some links:

http://guides.rubyonrails.org/index.html
http://github.com/trevorturk/static/
http://heroku.com

I think tptacek makes a good point, too, though. Learning how Sinatra works and checking out some example apps there might be a good way to start as well.
is it me or is Mint going to be the one success example that will always get thrown in your face as an example of a successful startup?
And Facebook has http://lite.facebook.com which is the sleek Firefox-like (when Firefox was sleek at least) answer to the Netscape-like bloated full version. Add to that "google-chrome --disable-images" and it's a nice text-only twitterish feed of what my friends are up to.

On a related note, as a non-computer scientist who is not working in your field who loves to read about computer science, I find the blogs referenced by reddit and HN to be shrill pundits who aren't interesting or academic. Most of them are on the same level as the pundits of the journalism world and often include troll/flame bait headlines which are designed to pit two camps against each other. They tend to be the Zed/Dvorak/TheRegister type people who all openly confess that they are trolling. The "academic" articles referenced tend to be mostly flamewars about comp sci education and practice that revolve around languages rather than algorithms or interesting implementations and libraries. I suppose the reason for this is that link sharing websites are often oriented toward leisure time and goofing off and so people are not in a "I want to learn a new concept" mode and just want to talk shop and gossip a bit. It seems that, despite all the critiques of the mainstream media that come from the digital generation, we do love this type of activity and unmoderated we actually trend towards it.

I predict any HN startups that seek to solve the journalism/media questions that PG wants to address will not get around this basic feature of human psychology and will not be able to become something remarkably different (although I am sure they have the potential to make a lot of money). Therefore social change of a more fundamental type will come primarily in innovations and disruptions of education rather than of media. However, the currency and lead that universities have both in terms of branding/prestige and state-sponsorship will prevent any major startups in this area and most innovation will take place in university IT departments deploying open source e-education tools. I don't think any e-education tools in the cloud will survive in the long-term even if they can out pace development from universities in the short term.
> $.10 / G

That is the inbound price, the outbound price (where you will more likely have an issue) is .17/GB
> The vast majority of computer science students pursue a CS degree because it has become a prerequisite for being a professional software engineer.

I don't know if things are different outside of Canada, but isn't this precisely what a computer engineering or software engineering diploma is for? That's what I'm doing and it's definitely different from CS - very applied, a lot of low-level programming (assembly, VHDL, embedded C), making a game with Java calling C++, source control is always used, etc.
The negative array index here is a bit perverse, granted. But the general idea of using structs (with ideally controlled, but hopefully at least well-known, packing) is surprisingly common, in my experience. I've seen it used for defining binary file and wire formats (which is perhaps somewhat dodgy but there you have it) and, perhaps more appropriately, for EEPROM layouts and the like in embedded systems.
Computer Science is to programming that telescopes are to Astronomy.

The concepts are what should be taught - not the latest buzzwords.
Absolutely. I loathe the emphasis on being an 'X programmer' in the software industry. I've never met anyone who described themselves as an 'Autocad engineer'.
I thought this was an announcement that they were supporting porting your number from other providers, but all it really is are new features and instructions for configuring your phone to bypass your carriers voicemail system and use Google Voice's instead by modifying the forwarding settings for unanswered calls. Still useful, but misleading.
I'm not really up to date on video games. I realize there is a lot of AI in the sense of smart opponents, but is there a lot going on with learning? I.e. enemies that get better at defeating you by seeing your tactics?

I've thought about this could be applied in RPGs... if you have a recurring opponent that learns how you fight each time you face off and gets better. Is anyone doing something like this (even at a basic level like "he uses magic a lot")?
this is kind of a big deal.
it always ended up with a few people doing most of the work...

This is a common problem with group projects in school. One of the main problems is lack of leadership. The solution: assign a leader.

I had a class consisting of 4 projects, 4 people per group, and each project had a designated leader which rotated. These projects were so much easier to manage, because the leader had some authority given by the teacher. He therefore had the ability to delegate responsibility and people would actually do it. In normal groups, even if a leader emerges, they will often be ignored by less motivated students, because their authority is much more tenuous.

My point: it helps to have an assigned (and verified by the teacher) leader for a group project.
It is like an erector set for grownups, makes me wish I had 300 extra dollars to spend.
So according to the results summary at the end, about half the adult population, and a quarter of college graduates, are under the impression that antibiotics kill viruses.

The other questions, despite being rather basic scientific knowledge, are arguably not relevant for most people, but that question worries me because of the antibiotic resistance issue.

It is kind of depressing that apparently around a third of college graduates think lasers work via sound waves, though. Sigh.
It's not as big as it sounds -- this isn't a scalable relational database service. As far as I can see, this is just EC2 instances with mysql preinstalled.
Fair enough. It doesn't really bother me if its called "AI" or "statistical learning" or "that thing that math guy over in the corner does" but I'm wondering what industries its used in?

This started as a general question, but since I've posted it I've been wondering if there is a market place for some generic statistical learning service. Something beyond just least squares regression, even if it doesn't use cutting edge ideas.
Peter Heather recently wrote a very similar history which I found to be very good: http://www.amazon.com/Fall-Roman-Empire-History-Barbarians/d...

Heather argues that the Roman empire was still internally healthy, but that increasing pressure from the Persians and the European tribes eventually overwhelmed the empire. Archaeological evidence indicates that the population densities of the barbarian regions may have increased by a factor of two or three since the early days of the empire. The population was larger and more sophisticated, and thus harder to handle.

But just this year Adrian Goldsworthy has written a book countering the new revisionist view that Heather and Williams present ( http://www.amazon.com/How-Rome-Fell-Death-Superpower/dp/0300... ). Goldsworthy argues that the barbarian threat really wasn't that much greater, and that the empire collapsed from within, due to civil war and political strife.

I haven't read Goldsworthy's book yet, but it's on my list.