skysight-inc/HackerNews huggingface dataset:
This scenario is reminiscent of what happened a few years back when AOL released their whole corpus of search queries with associated user-ID hashes. Most of what was done with that data set amounted to amateur CSI work and general meanness. I don't recall any profound revelations or can't-miss business opportunities coming out of it. The value of raw data is overrated.
http://stores.lulu.com/_why is gone too. :(
Man, he really went to town!
I'd like to see which search results were clicked on for a given search, and then associate those sites. And then do an amazon style: people who landed at this site also went to _____. As well as other relations based on user browsing, not based on site links.
I wouldn't create a business. I'd write a program that could learn (text-based) language. Then I'd write it up and submit it to Computational Linguistics. Then I'd die in obscurity as someone took my idea and figured out how to monetize it.
Don't Be an Open Source Douchebag
Twitter Developing Location-Based API
You have to consider that all that data isn't just a list of web pages. That includes information provided by the web page and the contents of the page itself (as well as associated metadata).

1) Create a map of the web (what links where and how), enabling an enhanced "browsing" experience (no more perusing a site for links to other interesting places)

2) The contents of those pages contain a large volume of technical documentation as well as a large number of opinions of the technologies that rose from the documentation. With both, and a large enough history of the creation of pages, the release of the documentation and the opinions presented, a model can be built to predict the success rate of any particular technology.

3) Given sufficient time, there is a large enough set of text that can be rendered to audio through speech synthesis (higher quality the better) in order to train a speech recognition system to a previously unseen level of accuracy.

4) Given a proper algorithm, sites with security vulnerabilities can be discovered just from what is crawlable and most likely should not be accessed. From that list, and given the total number of unique websites contained in the database, you can calculate the ratio of harmful to potentially non-harmful websites and provide a risk threshold of any given link on any given page.

5) Provide a search engine that uses regular expressions on different sets of the data (metadata, tags, text, text in a specific tag, etc.) to present a search engine with a previously unseen level of accuracy (accuracy of the results is dependent on the individual doing the searching, not the systems ability to guess that by std::list, I didn't mean "Sexually Transmitted Diseases STD List" (seriously, http://www.google.com/search?sourceid=chrome&ie=UTF-8... has that as the second result).

Edit: I failed at formatting
What I learned at the Lean Startup Workshop
Cheating? Looks like the Free Market at work to me. Ain't it grand, we should convert all aspects of our society from government, law, economy, health, environment over to the Free Market. It is that awesome! People with the most money and power should be able to use it to control the rest of us, right?
Along the same lines--a wiki that contains all documented arguments and counterarguments on a given topic instead of rehashing them all the time over and over again on social discussion sites.
DON'T RUN THIS SCRIPT. My IP was blocked from Hacker News and now I'm having to SSH tunnel in (and no, I haven't done anything unsavory on HN today) :)

Also, if someone could unblock 81.86.235.* I'd be much obliged, thank you. I have turned off this silly script.
The actual article:

http://www.theregister.co.uk/2009/08/21/exploit_operator_bro...
so you're saying that people can be controlled because they lack the foresight to understand that some reviews could be fake in any given iphone application? i think that's called ignorance.
Say what you want about AllThingsDigital but I find it ridiculous that a TechCrunch post offering a summary of the situation is linked to and commented on to when it doesn't offer any further detail or incite that cannot already be gleaned from the existing post and thread about the matter.
I would use it to fight crime/corruption. It isn't too hard to identify people from a few google searches. Examples:

Food places near X will give you a good indication where they live and their income
Find what they are searching during normal work hours. Often this will give you where they work.
Searches of themselves.

Then just use this information to hunt them down or expose them.
More specifically how 10 professional writers write.

Note some the advice is so general as to be useless. Plenty of professional writers write at night. Plenty of professional writers write in public spaces (just about almost any newspaper writer, for a start).

The observations made are interesting but don't really fall under "advice" any more than copying the writers' bowel or eating habits would.
I like this a lot. Sort of like Stumble Upon and other social discovery, but different, and implicit, versus explicit (via a Stumble for example).
Researchers create robots that can learn to lie and deceive, on purpose
AOL released the list of searches created by users, purportedly anonymized. You didn't have access to the index of their crawler. Also, their subset was really small. I'm talking about a continuously updated stream.

I have started learning about complex event processing (CEP) and am wondering if you could do anything with that as a model tied to suggesting related links/sites.
I stopped being OK with TechCrunch when they went after Blaine Cook personally for Twitter's reliability issues.

I'd forgotten about that. You're right, that was awful.

Edit: I like pop songs. At least Bleach had one.

Edit 2: Did you know that the WSJ is Noam Chomsky's favorite newspaper?
Understanding probability is not a reason to not vote.
Whether you should worry about your carbon footprint depends a lot on what your friends and/or family think of the issue. On a practical individual level, it's all about looking good in front of your friends and/or family.
(5) is an interesting idea. I'm stuck on what else in the meta stream you would want to search, but certainly giving the flexibility to search the tags as well could open up some interesting experiences.
I think that the date of Greenspun's essay is indicative of how little has changed, rather than how much. Most writers didn't make very much money then, and they still don't, which many people don't seem to realize, and they often work like astronauts to achieve relatively modest financial success, which people like the poster in the original HN thread might want to know before getting started in earnest at trying to write for the book market. Take a look at these posts: http://pimpmynovel.blogspot.com/search/label/knowledge%20is%... regarding current advances for most types of fiction.

"are you suggesting that the web has not panned out for writers and that they are equally screwed online as off?"

Depends on what you mean by "panned out" and "screwed"; I can't really tell from the nature of the question. If you mean, "Do I think writers can make enough from the Internet to support themselves?" the answer is yes; if you mean, "Will many of them do so, especially relative to the number who would like to?" the answer is "no." In fact, I even wrote a blog post on the subject of how unlikely it is for people to make money from blogging: http://blog.seliger.com/2009/06/17/youre-not-going-to-be-a-p... .

Still, I write this a wannabe novelist who is nonetheless going into things with his eyes open, rather than closed. I originally posted a link to this post: http://jseliger.com/2009/08/19/the-very-very-beginning-write... concerning advice for beginning writers who want to learn how to write well. But even if you do write well, you're not likely to be paid well for it, at least by trying to do so doing fun, creative work for the conventional publishing industry.
m/10 - m/13 > m/50 - m/100
> Religion and ignorance, in all their forms, are a curse upon mankind and should be mercilessly eradicated.

There is so much to know we are all ignorant. Some of us realize we are ignorant and some are so full of themselves they think they know everything.

You might notice that countries that ban or curtail religion want the people to depend upon the government (either do what the government says or starve). Any sort of organized Religion is the closest thing to a social safety net that is not government run. Until recently, even the United States didn't have a social safety net.

When I was in my teens and 20's I thought the same way as you. Then I became less ignorant. I know I don't know what I don't know and I don't go around pretending to know everything.
Though I must also add that I didnt really like RapidMind API very much. It was not very flexible

It sounds like you work in this space. I've become interested in GPU parallelism. What's the best stuff out there? And how do you recommend to get started?
Don't forget about the people you will be enabled to meet tomorrow (meaning the future) because of the choices you make today. That extra year of school opens up a whole new network of people. As I get older (mid thirties now), I am discovering how woefully I underestimated the value of "networks" for my professional life. I'm not talking about the fake facebook friends, or even twitter followers, but the bonds forged by the common experience of school. You'd be surprised how many people will take your call just because you went to the same school as them, even 20 years after they did.
"you find yourself competing with some insanely good writers."

Not only that, but you find yourself competing against writers who were alive centuries ago (Jane Austen, Melville) or decades ago (Fitzgerald, Hemingway). You also find yourself competing against many, many novelists in a fragmented marketplace, as I described in response to an agent's blog here: http://jseliger.com/2009/07/25/literary-fiction-and-the-curr... . Note in particular that you're also competing against a used book marketplace that's got decreasing friction.

That's not to say I'm complaining: but it's also how it goes, and people who don't understand the structure of the publishing industry or the forces acting on it might find some very unpleasant surprises as they wade into the riptide. One nice thing about Greenspun's comments about writing is that they're realistic and readily accessible. A lot of people who write about writing, especially on blogs, seem more romantic than knowledgeable.
It's not difficult to avoid the 'HR gateway' problem - just have HR concentrate on payroll, legal compliance issues, visas etc. & not interview or filter resumes. Interviewer should be the prospective recruit's direct manager, &/or someone who has/is or can do the job in question (wherever possible) and a seperate interview by someone else to confirm the first opinion. Is that rare? I've never hired any other way.
Actually, this might just be an important point in the discussion of what exactly "does or doesn't belong" on HN. It seems like all the highly-voted, well-liked, and highly-commented articles tend to discuss some sort of revelation of pattern, not revelation of fact. In other words, hackers like to read about new, effective abstractions (for seeing the world in general, not just for code.)
I wonder if they fell through, because in Kirigin's words:

"We strongly believe that social payments will work on a social network, provided that they’re done within the platform and not as a 3rd party."

So at that realization the acquirers decided to build rather than buy.
Raygun Gothic Rocketship counts down to its launch
There is another thread in this article asking how I would feel if this happened to a friend. The answer is "I don't care". I thought a bit about how this would make me feel if it happened to me, and the answer is "I don't care, because I don't use Facebook".

I then thought about what would upset me, and I realized that I would be upset if someone edited my blog and added random technical inaccuracies.

Then I realized that I digitally sign all of my blog posts, so this is mathematically infeasible.

(Example post: http://blog.jrock.us/articles/App::Persistent.pod/raw)
If you read the replies on the screenshot, you notice the OP's sister posts a message (granted, we don't know if it's for sure the OP's sister, but it is implied). The interesting part is that the last name of the "sister" is not blurred, whereas the OP's is. Maybe she's married and has a different last name though.
"Once you become intolerant yourself in opposition, you lose the basis for your argument."

Ah, that's the Achilles' heel of liberalism right there, isn't it? Actually, I think we must make an exception for intolerance. The goal is not just to promote tolerance at the level of the single person. It's to maximise it society-wide. With that perspective, intolerance is identifiable as a cancer in the system, and must be excised.

"taking positions that were far afield of what I wanted"

Ha. Sounds familiar! There's a fine art to maintaining one's argument, even in the face of in extremis counterargument pushing one further and further away from one's core position. It's an art I have not yet mastered. Elsewhere on this site you'll find plenty of examples of my being suckered into making spirited defences of unsavoury things while trying to defend some much more noble principle. Perhaps the format of this site's discussion is simply unsuitable to making complex arguments with deep roots, especially given the expedited timeframes for making a response anyone is likely to read.

A fews years ago I tried starting to write a book which would describe all my beliefs, and why I believed them, derived from as close as I could get to first principles. I abandoned the project when it became an organisational nightmare; trying to justify what I thought in one area turned out to have links to 10 other areas and after a while I didn't know whether I was trying to enumerate and codify my answers to moral questions or writing a blueprint for "my perfect society". Anyway it was completely unmanageable and I gave up.

I've given some concepts in graph theory and structured data a lot of thought since then and I should probably try again. Most of my beliefs can, I believe, be encoded in a directed acyclic graph. I should attack the problem again not as a book but as a web site, so I can just post a link to the appropriate section here and save a lot of time, while also keeping the discussion formal and free of scope-creep.

That would also satisfy your requirement for belief self-scrutiny. What could be more rigorous than explicitly enumerating and justifying - and thus throwing open to public scrutiny - every founding element of every belief you hold?

Damn, I really should get started on that ..
Surely it should be: 'I'm OK, but I've been in a car accident'? Then maybe mention the bull.
It took me five re-reads to figure out that you meant to type decreasing friction.
From: http://web.bsu.edu/cob/econ/research/papers/bohanon2002ir.pd...

Public-choice scholars have long argued that voting is
instrumentally irrational because the probability that a
single vote will change the outcome of an election is nearly
zero. Dennis Mueller made the point well when he noted
that "the probability of being run over by a car going to or
returning from the polls is similar to the probability of
casting the decisive vote. If being run over is worse than
having one’s preferred candidate lose, then this potential
cost of voting alone would exceed the potential gain"
(1989, 350).
Absolutely. The jQuery test suite still uses the original markup from your script's test suite. I wrote my selector implementation at about the same time that Dean Edwards wrote his (but he published first). You were the absolute first, though. Major kudos to you, for sure.
Yeah, TechCrunch is getting like this too, so I've stopped reading both of them. Nothing really wrong with that, just not what I want to read. Mostly for me it's now HN for tech and VentureBeat for funding related stuff. Oh well, things change and you can always vote with your feet (or eyes in this case I suppose).
Musical sorting algorithms
Start the world's most underhanded (and successful) SEO firm.
I don't think this matters all that much. There appears to be very little correlation between reviews/ratings and sales. This is especially true at lower price points, which is pretty much everything. Apps might be different, since I'm basing this on conversations with game developer friends (Touch KO, Wurdle, others).
Sorry: fixed it in the original. Thanks for the heads up.
If you already have "competitors" (that is, if they see you as their competition), you're past the idea stage. If you don't, then they're as likely to listen to you as to an anonymous email telling them to do features X, Y, and Z.
While I think that breaking into people's stuff and vandalizing it is horrible, in the specific cases I've seen(such as the one in the article), no actual harm is done. All the victim has to do is change their password and inform people that someone else obtained their password and that they were not the ones who posted the untoward messages and photos. If the attackers have permanently taken the account by changing the password and email address, that solution is not possible, but then it's not really the scandalous posts that are the issue, is it?
Contrived?,yes, useful?, yes (at least in my experience), do i plan to use that formula in every interaction?, i don't think so, it wouldn't be natural.

Thank you for sharing the example of the kid who "rained all over himself", it clearly shows that exceptions apply.
> When I was in my teens and 20's I thought the same way as you. Then I became less ignorant. I know I don't know what I don't know and I don't go around pretending to know everything

But that's not the case with most religion. Most religion says, "We know everything. The answers to everything in life is $DIETY!" Look at all of the Evangelicals that run around trying to force their religion on everyone else, and becoming 'soldiers for Jesus' or whatever it is they do in the 'Jesus Camps.'

I can see your point with religion being a 'social safety net,' but that is historically true because it's the best way to bolster membership. When people are down on their luck and maybe feeling like the world is against them, $DIETY's followers are there to help pick you up and tell you that $DIETY is the meaning of life while they help you out.
The future is certainly hardware-neutral APIs such as OpenCL and DirectX compute shaders. However, for learning, its best to start off with Nvidia CUDA because there is a ton of material for CUDA online and CUDA is also available here and now. Once you learn CUDA, switching to something like OpenCL is easy. For production code, there is no longer any reason to write CUDA.

And also a word of caution for new to GPUs: dont expect miracles from GPUs. Set reasonable expectations. GPUs are not magic.
All the academic writing I produce and read begins with a conclusion, in the abstract.
The problem is that when people get into things that hard they tend to only 'stick to their own kind.' It's really easy to be sucked into things really hard like that when you're only around other people that believe in it that hard. Why do you think that most cults have compounds, and that in some cults you're not allowed to talk to outsiders?
A small device embedded in a car's dashboard that monitors events such as:
- Anti-lock brakes or traction control kicking in (slippery road conditions)
- Drastic changes in temperature/humidity (potential for ice, fog, or whatever)
- Current speed (traffic flow)

and uploads this data, along with GPS coordinates, to a central server via cell phone data link or WiMax or something. Aggregate data about traffic speed and weather conditions nearby could be sent out to subscribers for display on an in-dash device:

"Hey, I see that 1.3 miles ahead on Hwy 494 three cars reported traction control kicking in within the last 20 minutes. Be careful!"

Getting manufacturers' cooperation for hooking into the cars' computers would be really tough. But the technology is perfectly feasible.
I often think of SMTP as well -- robust message queueing and routing with built-in failover, backoff algorithms, and myriad tools to view all the parts involved. An RFC822 message's envelope can contain infinite metadata and its body can contain text, json, xml, base64-encoded binary, you name it. You can encrypt messages with PGP, or secure the transport layer if you can determine who talks to you. There are more implementations that one can count, it works on every platform, and it doesn't scare away the systems guys.
SEO? With something like that you could programattically find semantic locations with lower coverage and then sell the knowledge that you could possibly attack that keyword. For instance, I'd love to see a chart which plotted frequency of use in an English corpus against some hypothetical Google-coverage variable. Anything that's a strong outlier from something like an exponential curve could be a good target.
I'd like to know more about reasonable expectations. What sort of algorithms are GPUs best for? I'm interested in using them for computation, not actual pixel shading. If you could give an example of something that would be in the sweet spot for GPU parallelism, and something that wouldn't, I'd appreciate it. Or a pointer to some good sources.
No Matt, it's not that. It's writing up a story, not having received comments from one party, explaining how you've given the party many chances to comment on the matter, for honesty's sake and the integrity of journalism, yet never considering that perhaps that party's refusal to comment could be due to other matters, legal or what have you, and indirectly making that party look shady for not returning the honest gesture.
> given the expedited timeframes for making a response anyone is likely to read.

I think that you just need to not be so concerned with this... Unless, of course, you're talking about taking a week to write something out. If you're talking about "if it's still on the frontpage," I tend to use the 'threads' link at the top to frequently check on any responses to my posts.
/b/tards is what 4channers call themselves
And just yesterday or 2 days ago I wrote this under the story of TipJoy shutting down:

"The whole story seems to repeatedly wave the arms and shout:

"Hey, Twitter! Hey, Facebook! Here we are! We know how to do this! Hire us, hire us, hire us!!!".

At least that's how I read the story. And I'm quite convinced it's their intent. Which shouldn't be a bad thing too.

Well, of course customers' money is the most important thing, no doubt about it.

But it struck me that you mentioned several times in the article that you know what and how to do, you just don't have the leverage of being the main body. Why would you deny it? Is it something bad? I don't think so. Would you refuse if Twitter or Facebook approached you with a decent offer?"

And for this I was severely downmodded. Well, wasn't the part of Facebook approaching you right? In my opinion you should have made this known to give people the full picture. For them to judge and make opinions having all the facts, not some of them.
It's really tough. New Mogul worked because it was started by Nickb, a virtual link submission machine. For a while it was just a lot of stories submitted by Nickb and he mostly voted up every story submitted by someone else. It was only about 2 months ago that there started to be lots of comments. About a week ago Nickb went on vacation and hasn't been submitting and the site has held together.
You know that bit in Office Space where they're asking each other what they'd do if they had a million dollars and one of the guys replies that he'd invest half in mutual funds and take the other half to his friend who works in securities?

You're that guy now :-)