skysight-inc/HackerNews huggingface dataset:
Our service, 80legs, will let you easily do this. We let you specify seed links, how deep you want to crawl, and control many other aspects of the crawl. By default, we control the hard bits, like redirects and spider traps, but if you want to override our default functionality you can easily insert your own code to do it.

Our default functionality will let you identify mp3 files by regex or keyword, but if you need something more sophisticated you can override that too. I'm pretty sure, based on what you've said, that you could simply put in a few parameters and start running some jobs within a few minutes of getting started with 80legs that will do exactly what you want. If not, adding custom code to 80legs is pretty simple too.

Just send us your contact info on our website (http://www.80legs.com) and mention HN and I'll make sure you get a beta invite. BTW - we're still in private beta and the service is still free for right now.
Cities in the US with the most female PhDs
Tips for College Students - Maximizing Efficiency
43654
It's sad that women never managed to log in... but out of 30, there couldn't be too many women in the application pool to begin with.
Laos is actually WAY off the beaten path.
I'd recommend against using "kick-ass" on the homepage. Save the attitude for the blog.
The fact that Zed wrote Mongrel and got no recognition is possibly an indictment of several things: the RoR buzzstorm, the Rails community, the “OMG Ruby is the new Java for Web 2.0″ technorati, maybe even venture capitalists. But it is not an indictment of the BSD license.

Here's where I stopped reading.
The blog post I wrote wasn't primarily about the legality of scraping (and I also didn't expect it to be read by more than a few people). But as that seems to be the topic of the thread, here's my response.

The courts found that it isn't possible to copyright facts, and that's all we were scraping - things like addresses, business name, and phone number. We weren't even scraping things like business category, because something as simple as putting a restaurant in the category "Fine Dining" might be considered a judgment call and therefore value-add by the original site.

And think of what would have happened if the court had found otherwise (i.e. had found that lists of facts could be copyrighted). If you opened a store, and I was the first one to put your address and phone number on-line, no one else could ever include your address or phone number on their site. Even if you created a website for your own business after I published your address, you wouldn't be able to include it on your site, because you'd violate my copyright.

I can't see how the Supreme Court could have ruled any other way.
You MIGHT be right, but I do web application development for government, and I was recently surprised when my new laptop was delivered with IE8 on the standard image.

For this particular agency, EDS is their GSS provider, so if EDS is incorporating IE8 into their ESA image, I wouldn't expect that it's too far off at other agencies they provide general support services for.

Of course, now I've got to make all the apps I've written work in IE6 AND IE8, alongside Firefox, which they're also piloting. I'm pleased at their progress, while staying wary of all the work it's going to bring my way.
I've done a little bit of open source work (a few small numerics libraries and the beginning of a small project in PHP) and I always release my stuff under the BSD license. To me, the GPL just doesn't seem 'free' (to reiterate what embeddedradical said earlier).

All of the stuff I've written has just been to fill a small hobby and because the only offerings out there were all expensive non-free libraries that I couldn't afford to pay for (not for a hobby, anyway).

I seem to remember that the very bottom of the networking stack in Windows was (and may still currently be) based on a BSD sockets implementation. If Microsoft had implemented their own methods from scratch, who's to say that things would be as compatible as they are now?

I respect that some authors want recognition (or perhaps a job) as a result of the hard work they put in (for free) on a lot of open source projects. But I think software is much more useful under a license that allows people to do whatever they want with it, commercial or otherwise.
Most distributions are GNU/Linux -- ie, they use the GNU userland tools (gcc, bash, coreutils, sometimes Gnome) and the Linux kernel. GNU/Linux is the most popular OS based on the Linux kernel, but there are others. Android and Chrome are two examples of non-GNU operating systems based on Linux.
If you look at many of the Tory policies, they're very very much to the left of the Republicans and even many Democrats.
Microsoft announcing details on Azure
Trent Reznor on User Experience
My first thought was, "Damn that's pretty."

My second thought was, "Nice AJAX, and it works..."

Very polished job. Nice work!
This response is solid, however I don't think the OP suggested that "True science is political." (Although it's hard to tell when people don't express themselves clearly.)

There have been many scientists to have admitted that they have kept their mouth shut when they oppose the status quo for fear of losing jobs or failing to gain funding. The concept of "a few bad apples" is certainly not new.
ytNumbers,

Thanks for trying it out. I need to fixing the login redirection. For the meantime, please login first (http://www.launchset.com/user/login/) before going to Link Accounts (http://www.launchset.com/user/admin/linkaccounts/).
The only cities we have good data for are Prague, Copenhagen, and lower Manhattan. Try http://www.nelso.com/cz/prague/
Indeed. There's an okay discussion going on at New Scientist, where they talk about how we can liken a human baby to doing the exact same thing - getting what they can (food). So I think we can use a better word than "exploiting" which has negative connotations - it's more based on survival. I must say though that when I put down the wet food, I certainly get a different cry, likely because they like it and I give it to them only when I think it's appropriate (rare) - not when they want it. They do run out of dry food daily or every other day, but they don't call for it, or cry for it... they'll come running when the bag or box is opening, but it's like "aww.. dry food again!? I guess we'll live".

Baby pups at malls or breeding grounds are simply too difficult to ignore. ;)

With both of these species, going back to the Egyptians for cats, and not sure when for dogs, over time they've evolved to be alongside us - and sheesh, we have kennels, animal daycare, grooming, the Wag Hotel in San Fran, and a new airline that is in talks to be only used for our pets?? Isn't it our fault as humans for the "exploitation"? I don't think it should come as a surprise, but the media putting a negative spin on it isn't necessary I think...
I suppose, given that Safari, while being an excellent browser, on Windows it is not such a great experience and just feels weird, it is reasonable to not list it as an upgrade option for IE6 users. By the nature of the situation, this only happens on Windows.
I've wondered that sometimes. I think it's because a lambda opens to the left, and starts with "l". Rhonda opens to the right, and is a similar word starting with "r". It's not supposed to make sense; it just cracks me up :P
It's an alternate browser that supports modern web standards. How is it any more 'pointless' than Firefox or Chrome?
Hurd didn't boot until 1994, and didn't really pick up until 1996.

http://www.gnu.org/software/hurd/history.html
while there is one (annoying) search result that points to www.experts-exchange.com stack overflow is relief.
GP takes down repair time for software bugs from more than 3h per bug to 3min
>How do you find sufficiently qualified and trustworthy partners who are willing to remain on standby for tiny one-off tasks like these?

You spend time up front to find them. With a few hours of up-front work it's not very difficult.

>Moreover many of his bullets just make no sense whatsoever. How am I supposed to outsource "Fixing Post-Launch Bugs" (on my code-base)? Creating Unit Tests? Database Development?

We've outsourced post-launch bugs to a very talented developer who works for us 2-8 hours per week depending on the bugs we find.

Creating unit tests is one of the easiest development tasks to outsource, and you'll find out pretty quickly whether your object model makes sense when you assign this to someone who didn't build the objects themselves.

>I have personally witnessed more than one startup fail spectactularly because the CEO drank a bit too much of the outsourcing kool-aid.

But that doesn't mean outsourcing is bogus, or that it doesn't work.

>it still takes quite a bit of time to bring a contractor up to speed, even on small tasks.

If you're working on a credit card processing system you are correct due to the complexity. But most apps are not this complex, and you'd be surprised how quickly a solid developer can start fixing bugs on a system they've never seen before if they are talented and the architecture is sound.
wind, solar, tides etc!

The article shoves these aside with an 'we currently can not make the required quantities using these technologies so we have to choose between coal and nuclear'.

The problem is not to switch to coal or nuclear today though. So let's keep those windfarms coming, better and lower priced solar cells and let's see what other truely renewable energy sources we can find.

Then let's solve the remainder as much as we can by burning waste rather than nuclear power or coal.
If a game offered the immersive, cinematic experience that a movie does, then yeah, I'll easily pay $10 for it. The Mass Effect game for example is awfully tempting.

But as it is that is not what most iPhone games are. Most iPhone games are time wasters so you're not interminably bored at the metro station - that to me is not a high-utility use case.
Hm, wouldn't plain checkboxes suffice? That's a pretty entrenched convention. Not to mention that using stars in that way doesn't really mesh with that convention either. Is there some aspect of it I'm missing?
:) Yeah, that's one I've debated a lot about. It came about when explaining the site to a designer who was working on the logo. It was getting wordy trying to explain the concept through text and I just ended it by saying "In other words, you don't have to be a programmer to have a kick-ass website." She latched on to it and when I got the logo back, it has the tag-line attached. I kind-of liked it, so I kept it...for now.

The funny thing is that I've had mostly great feedback from it. It makes people laugh a little and they tell me they think it's great. It's not the best marketing material for business users though, I agree.

I thought it might capture a bit of what we're trying to accomplish. To make users feel comfortable and I can imagine a lot of them saying "Why do I have to know all this programming stuff just to have X on my website?".

Thanks for the advice. I'd love to hear some other opinions on this from the HN crowd.
It looks to me like you're complaining that the author got too close to something you can't say. He stated the sample size and some interesting observations, and specifically pointed out that he's not trying to claim causation.
It's not so much as adding, as making it readable and interesting.
There could be up to 27
That was the point of the post. Without some manual oversight of every new listing, too many errors will creep in to make the database truly useful.

Local search is not like general text search, where one expects that many of the results returned are not relevant, are wrong, are spam, etc. and can be ignored. Having the incorrect address or phone number for a business on a local search site is a major irritant to the user, and will quickly erode one's user base.
Do you have an api or do you support some desktop/console linux clients? I prefer having a blinking icon in the taskbar when there is some non urgent activity demanding my attention and checking a browser tab every few minutes is inefficient.
Yeah, myself as well. I don't understand this bitterness that comes from the Python community. Or maybe, it's just a vocal minority. There was no need for this in the article.
The freedom to use it in commercial software is restricted by the GPL.

Not at all. Software licensed under the GPL cannot be used in proprietary software. There is no restriction on commercial use.

For this reason I consider the BSD/MIT licenses more free, for it says nothing except, go ahead use it, tinker with it, change it, republish it, whatever.

Again: "free to whom?". From the user's perspective, the GPL provides more freedoms than the BSD and MIT license.

The GPL is a legal tactic of the FSF movement, who believe that intellectual property in software is immoral

This is not correct. The FSF is against use of the term "intellectual property", because it conflates unrelated sections of law, but they are not against copyright. The FSF site is copyrighted, their licenses are copyright licenses, and they even require assignment of copyright for any patches applied to GNU software.

I do not understand where the belief that the FSF opposes copyright springs from, but it has no basis in reality.

The BSD/MIT licenses are more free for the user, for they can now use it in commercial software, a right eliminated by GPL.

That's an odd definition of "user". What about the person sitting at the terminal, using the software? Are they guaranteed the right to inspect, modify, and distribute it? Remember that Stallman launched the FSF out of frustration with a poorly-implemented printer driver that he was not allowed to fix.

Every right granted to somebody is taken from somebody else. The emancipation proclamation stripped rights from slave-owners. Women's suffrage stripped rights from men. The GPL strips rights from distributors. Nevertheless, these things are named as increasing freedom because the rights they remove are ways to hold power over others.
Yeah, at first I was like "another free host, WTF." But then I realized that you're really closing the "last mile" of open source web site deployment and hosting by automating the process. It is pretty ambitious.

One benefit to the user is that the add-ons/ plug-ins are sort of "Flooha-certified" which is helpful as there are a lot of junk/insecure modules floating around out there.

Another cool feature would be the ability to auto-migrate between different CMS.

But, what is the revenue model in all this?

Good luck and keep us posted!
They don't have anti-discrimination laws in Thailand?
Money down the drain...
RedHat also gave him some fat stock options, just to say thanks.
[joke alert] `Ignore the Clouds, It's Sunny' would be a great title for an article about Cloud Computing. Too bad this article deals with social life...
What browser are you using? I'm on Safari 4 and the page looked like this http://i31.tinypic.com/mvmgx0.png until the font was downloaded (took almost 10 seconds on my slow connection).

Edit: I tried it with Firefox and it does indeed use the default font until the new one is downloaded. The transition is not nice but better than nothing.
Funny idea. That might actually work with the new 3.0 APIs.
At one level, your question contains the seeds of its own answer. Strong GPL partisans tend to assume anyone doing things where the GPL language/intent is fuzzy is "doing something obviously questionable".

So one fear is, you could wind up on the right side of the license and the law, but still face resentful, accusatory opinions from parts of the community. And thus a related fear: your project could be distracted by long discussions like these.
Ya know, this brushes against two things we've been doing/talking about.

In our experience, the hardest (technical) part of developing this product has been notifications. There is no good way for background web apps to notify you, especially visually, when there is some event that wants your attention. Something like growl for the web would be awesome.

Anyhow, we have plans for a public API (in fact, we've had a bot in our test channels before, exercising some early versions of it). Unfortunately, it's not ready to roll out quite yet. We know it will be important for all kinds of custom integrations.
A Look Back at a Month of Binging
The "floating popup" navigation could use some more spacing between the letters.

On the signup page there is "Capcha" (with no noscript), it should be "Captcha" (I think, that word is terrible).

No other comments because I did not check out the rest.
Engine Yard Programming Contest - Win an iPhone 3GS PLUS $2k in Cloud Credit
We’re kicking off a programming contest today that is sure to challenge even the most comp-sci heavy engineers out there, and we’re excited to see what you all come up with. With the difficulty of the challenge in mind, we’ve got some great prizes for the winner: an iPhone 3GS AND $2,000 of Cloud (Flex or Solo) credit.

You must tweet a sequence of twelve words that when hashed is bit-wise closest to a hash of a challenge phrase that we will announce the morning of July 20th. All words must be from a 1,000 word dictionary we will provide at that same time. You are allowed to append up to five random characters to the end of your entry. We’re pretty confident you’ll want to write a program to automate the finding of close matches, so announcing this a week in advance should give you enough time to get your programs up and running.

See the blog post for more details, rules and prizes!
Could you expound why? I had a boss who hadn't written a line of code in his life ask me if we were using RoR around 2006...

Edit: what I mean to say is that the author is correct that there was a lot of baseless Ruby fanboyism.
could be handled by directing people to a posting about hacker news, where they could comment to their hearts content (or, hopefully, read historical conclusions to their comment's topic, if one already exists)
Was using FF3.5, sorry for not pointing it out earlier.
Not for the sites I generally interact with. I'm bandwidth limited 80-90% of the time (and that's with a 3Gs).

I'll agree that rendering is slow as well, but 3G, edge or worse (the O!) have horrid load times when grabbing scripts from pages I like to visit. They fly (in comparison) when I have wifi access.
Cool contest, but starting it on a Monday morning sucks pretty badly.
It struck me as off-note as well. I mean, if you are investing in a web host there needs to be assurance that it is a serious effort that won't disappear and take my site with it.

Having said that, I ate dinner at a popular joint called Ray's Hell Burger last night, so maybe attitude is the right way to go!

A bigger issue in this vein is lack of pricing-- I'd feel better about the service if I had to pay a monthly fee, or if there was at least a freemium model so I'd know somebody was subsidizing my free kick-ass site.
I'm afraid I don't see the point of this (very brief) blog post. A good "plan" is not simply a wild ass guess at how the future will unfold. It also covers, directly or indirectly, the past (industry trends, experience of the participants etc) and the present (competitive landscape, financial and other assets etc). Even when addressing the future, a good plan will be informed by envisioning a diverse range of possibilities that may or may not unfold.

And of course, a plan is not really a roadmap so much as it is a way of marshalling resources (money, courage, etc) to begin a journey into the unknown.
The GPL is ultimately about ideology, not software. Note all the comments by GPL-boosters demanding to know what kind of unsavory license-bending the poster really wants to engage in due to his asking a few simple questions with a distressing lack of clear answers.
I don't understand treating nuclear waste as such. If it's radioactive, it means it can still be fuel. Am I missing something? As far as I remember, the plants in France reuse their waste until it's almost completely non-radioactive and then purpose it to industry.
In regard to the 5 point Neil makes under the original post, I have a sad observation:

- numerous people would, and often do, rally against gag orders handed down by various offices

- yet the wide society is pretty quick to gag or chicane a blogger for publishing politically-incorrect observations. Observations that were surrounded by disclaimers of small population sample and other factors that clouded them. And while it's not necessarily the same people as above, it's still the wide society.
Interesting, I'll look into it-- we haven't experienced that problem yet.

As for the NYT feed, thanks for the suggestion. I'm on it!
Well Verizon and AT&T now own the spectrum that was once used for analog TV. LTE should be able to sustain and provide the level of bandwidth needed for this.

A Clear channel or iBiquity (HD radio which is boring and needs a Pandora to jump start it) could be potential buyers of Pandora. I would say those who invested are looking at such entities as potential buyers. Also, another potential reason why Pandora is jumping onto RIAA ship saying clear channel should pay the same rates.

It may not be feasible now with 3G, but 4G and other future wireless options will be.
Too much meta is bad, too little meta is also bad. If you live by 'meta murders', you're going to become a braindead doer, instead of a creative and insightful contributor. I believe that to find a good balance of meta in all things, we should simply look at meta-cognition and find balance there, and then extrapolate out.

I recommend checking out this article from Seed Magazine: http://seedmagazine.com/content/print/thinking_meta/ :

Unless we are aware of our tendency to act irrationally in certain situations, then we’ll continue to act irrationally. That much is predictable.Although the mind is full of flaws, we can learn to outsmart them.

I think for an approach to productive use of meta discussion in these tech circles, we should begin by looking at the already existing strategies for utilizing meta-cognition for our benefit - like the ones listed here: http://coe.sdsu.edu/eet/Articles/metacognition/start.htm

In short, thinking about thinking, thinking about you think, are quite valuable, but do it wrong, do it too much, and you end up driving yourself nuts -- and say crazy things like meta is murder and run away from it....'I will not think, I think too much, will not think...will be braindead zombie not questioning what I'm doing...' -- also, your analysis of your thoughts isn't often that valuable to other people, which I think relates to why we find reading about bloggers telling us how they blog is kind of mmm, boring.
There is no need to build a large number of Nuclear or Coal plants any time soon so the question is almost irelevent.

As a side note: Why do they feel it's nessisary or even reasonable to find people with strong vested intrests when looking at these topics?

researches capture and storage of flue gas CO2 and other fugitive gases
studies clean coal at the University of Queensland.

Not that finding people who don't understand the toppic is all that much better.

is a novelist, writer, and former fiction editor at The New Yorker.
I tried out and stopped using the JoS discussion boards (way before the days of reddit) primarily because this release valve didn't exist. My submission would occasionally get magicked away and there was no recourse, no way to ask why that had happened.
I think a better term is Self-Referential

Meta data is not always self referential.
Heck, if we advertise for a Unix C++ programmer, I think half the resumes we get don't even show C++.
Bad, bad, bad plan. Sometimes i wonder if 37signals exist in In the real world :)

If I go to my boss with a "5 year strategy guess" he will fire me. On the spot - and this is a guy who doesn't do corporate crap either.

Guess implies a level of ignorance in your data. Plan implies less ignorance and more consideration.
Who "trusts" bit.ly or tinyurl? I trust them about as much as I trust http://.
NSA's cyber overkill