skysight-inc/HackerNews huggingface dataset:
Naturally, Bing is hosted on a Windows server, which inherits the Windows filesystem eccentricities. Case insensitivity is among those. Because "search" has six letters, that would mean that the robots.txt would need to have 64 entries to completely exclude this directory. That's not even including the tilde thing or any other paths to that directory. And that's for one directory.

Lame? Yes. Google's fault? Not in the slightest. But it brings up an interesting question: if MS clickstream gathering included an opt-out mechanism that happened to be impractical for Google, would that change the ethics of any of this? Say, by having the Bing toolbar identify itself in user-agent so that Google could block it if they wanted?

I wouldn't think that would materially change the situation. If Google really wanted to, they could probably "block" this now by encrypting their existing URL redirects, thus hiding the URL from the Bing toolbar entirely, at least until the user is out of the Google system.
To be funny, O'Reilly parodies need to get the design right. That one is too far off in the design, making it just embarrassing. In particular, the title font is too small, leaving it lost in a sea of white.
Apple was obviously not involved in the development of this app (except for providing News Corp. with early access to their subscription API). Why would think they were?
Because anyone concerned about the safety of something being forced on the population is an anti-science religious zealot, right?

Seriously, that's what this whole thread is about. If you are worried about whether vaccines are safe, then you want to "kill children".

I think the burden of proof is on those who want to force everyone to be vaccinated to prove that none of these vaccines will ever kill a child.
Yes, nothing exists on the internet if it's not on Google. When people click on things in Reddit, if Google hasn't yet indexed it, then it doesn't exist.
I saw cofoja (http://code.google.com/p/cofoja/) this afternoon, and realised it would be good to be able to code like that, but since I spend my days doing mostly Javascript I wouldn't get to use it. So I've basically just ported it across. It's just an afternoon's work so it's probably incredibly bug ridden - don't even consider thinking about using it for real production code!

There are some other libraries that do the same thing for Javascript, (http://www.cerny-online.com/cerny.js/ http://weblogs.mozillazine.org/weirdal/archives/016921.html and one also called jsContract (oops - - looks like I might have to change my name...) http://kinsey.no/projects/jsContract/)

The difference with these is they all expect you to call a special library within your functions. My library takes the cofoja approach, where you keep your function the same, but wrap it up with annotations that specify the contracts. You can even turn contracts off completely, by setting a variable.

It has some tests there, but they aren't in jsUnit - I'll do that tomorrow. My next step is to put contracts on each of its own functions, just to prove it works.
It would be equally unethical to be copying DDG's results in this fashion.
Finally....something useful from Facebook!
Nodester is a super cool project, because it is a node.js hosting platform that is fully open source. I am preparing a similar node.js and couchdb hosting site (http://cloudno.de) based on the work of Chris and Dan. I guess there are some more coming and the long wait for a coupon/invite will be soon over.
Yep, I wrote a bunch of C macros to prevent them and use them everywhere:

curl http://zedshaw.com/dbg.h

The main thing that caused defects was not handling default cases in logic. Things like not filling in the dangling else of an if-statement, not trying to return a value from a function no matter what, and not having a default: in a switch. By always trying to include at least a comment there, or an assert it forces you to think about the logic of the whole statement and prevents bugs galore. Combine that with unit tests and you've got a good solid start.

I don't recommend it because it's an insane amount of work above just writing code. I think it'd be something I'd force on high speed students in a kind of "Boot Camp" situation, but not on anyone else. In fact, I think most programmers couldn't handle having a giant bag of evidence staring them in the face telling them they suck. Also I think most programmers wouldn't get the stats behind the analysis and just keep screwing it up.

I play guitar when I have to think about stuff. I take one to work, and when I'm stuck I bust out my guitar, a pen, and a note pad. I then noodle and practice, and as ideas come into my head I write them down without filtering.

Basically, you need a hobby. :-)
Has anyone made a plugin or anything of that sort which abstracts away the grunt work of using this sort of system? It sounds like a great thing to try out were it not for the opportunity cost of trying it.
very intense! The Founder is a real efficient startup program.
You'll have a powerfull network, clever mentors, and cool peers.
Don't hesitate to enroll; yves laurent kayan klymba@gmail.com
There also exists the ControlMaster configuration option which treats multiple sessions to the same host as a single connection. With this enabled, you only have to enter your SSH passphrase once per host. Aside from the convenience of not having to repeatedly enter your passphrase, subsequent sessions are initiated much faster (due to not having to renegotiate) and there is (theoretically) less strain put on your network adapter. Note, however, that if there are no active sessions to a host, the connection to that host will be dropped, and you will have to re-enter your passphrase upon reconnection. See the ControlPersist option for dealing with connection persistence.

If you're like me, however, you'll use both keychain and ControlMaster (for the increased negotiation speed) :)
I think early adopters are smarter than that. They don't need to be lured in with the offer of free stuff. The very reason they're early adopters is because they grasp the potential for a new service and how it fits into their lifestyle. Not because of a superficial offer of freebies.

Companies that are confident about their quality products at an early stage don't need to bait beta testers with free stuff to lure them in.

In fact, I think you're going to skew the kind of user who would sign up based on getting free stuff in the wrong direction.

No doubt, you could get _more_ users if you offer free stuff, but at this early point, companies want the _right_ kind of users, not quantity.
I cannot debate a scientific issue with someone who rejects, out of hand, the entirety of the scientific process, as you do. When you use the term "now-settled science" you are rejecting the scientific method and science.

Only in the realm of politicians and ideologues is the term "settled science" used.

Since your opening position-- and that of the mandatory vaccination ideology-- is to reject science and claim that you don't have to be accountable under it because you claim your opponents reject science-- you have already lost.
Are we reading the same file? Under where it describes matching paths (so replace /fish and /Fish with /search and /Search if you like):

==Example path matches==

[path]
/fish

Matches:
/fish
/fish.html
/fish/salmon.html
/fishheads
/fishheads/yummy.html
/fish.php?id=anything

Does not match:
/Fish.asp
/catfish
/?id=fish

Comments:
Note the case-sensitive matching.
Everyone else in this thread has already explained that microsoft is not properly using robots.txt by having case insensitive urls, hence why these urls were indexed.
Goldacre is great, but don't disparage Gates as not having a relevant background. He's a really smart guy, has been involved in vaccination efforts for a long time, is known for trying to spend money effectively, and has pledged to spend $10 Billion dollars on childrens' vaccinations in the next decade. He's being quoted here as an expert and not a celebrity.
There are lots of ways to point out the superiority of a product without insulting people. Anthropomorphizing computers is obviously designed to lead people to connect the machine to the person, making the step from "choosing a PC is stupid" to "people choosing a PC are stupid" very short.
Pickaxe factory factory factory is where the real money is. [1]

[1] http://discuss.joelonsoftware.com/default.asp?joel.3.219431....
This is not unique to The Daily. Popular Science and Wired are exactly the same (maybe others, too). (This is not an excuse. They all suck equally in that respect.) I suspect that they all use some sort of InDesign plugin or something similar.
I have used it for a web statistics/analytics site. The combo is killer. And CouchDB's group level map-reduces make the data aggregation a snap.
I'm not sure if I agree with you, but I really do appreciate you making this point so we can openly debate the merits (and pitfalls) of this perspective.
There is absolutely a reason. A user queried for a string, then followed a link. Biasing Bing's search results towards the followed links is a signal that improves their search.

Yes, but the actual effect is that they've just copied Google's results, rather than extracted valuable insight from their own users. And I don't see how Bing could have scraped the search terms and results from a Google session unless they are using code specifically tailored for Google's site. Given that, the clickstream excuse rings awfully hollow.

Plagiarism is going to be a judgement call in a developing field like search. Google's judgement is that Bing's scraping of their site, in this case, is fairly underhanded. I would agree.
Of course I'm superior to Mac users, I'm a Linux User. (Yes, this is sarcasm). I do think I made a better choice, that's all.

I highly doubt that I'm unable to conceive of the possibility that people in government are greedy. I just finished saying that greed is a base instinct that nobody is immune to. Which is why regulated capitalism and constitutional democracies are such nice compliments -- you get a chance to boot the crooks out every 4 years or so. It's far from perfect, but better than the alternatives.
Good points, like I said - horses for courses (the right thing where it fits in other words).

My IT teacher at school instilled the notion that we shouldn't always take a computer based approach but look at the best tech for the job, even if that is pen+paper. Sometimes a spreadsheet is the best db to use, sometimes ascii is a better image format (rarely IMO), etc..
Android Market Could Open Phone To Hackers
No. Even a good idea is worthless unless implemented. This is actually the complete opposite of that.
Introducing variety into my routine always improves my programming skills.

I started in high school writing homework helpers in Java and C++, and writing graphing calculator scripts to make fun patterns. Then I worked with other programmers, and started structuring my code to make it understandable to others. Then I started ACM programming competitions, and learned to rigorously solve problems. Then I found Python, and learned that finishing can be quick and easy. Then I got an internship at a major corporation, and discovered I never want to be an enterprisey programmer. Then I got a GSoC gig and learned to collaborate remotely and make estimates. Then I got a real job and learned to work with code for months and years, instead of churning out small projects, and the importance of finishing early and often. Then I experimented with Lisp, and learned the importance of abstraction, community, and salesmanship. Then I played around with Tornado and started learning about the web. Recently, I got a better job and started learning the importance of testing, the importance of listening to users, and the value of a second set of eyes. Recently, I've started experimenting with the Android SDK, learning the power and limitations of major constraints.

I can't wait to see what the future holds.
What do you think of Minix 3?
More on the retraction of the Salon.com article http://sethmnookin.com/2011/01/17/salon-retracts-deadly-immu...
The truth can suck -- and also sets you free. I embrace it. It's a big part of how I am getting well. But I am much less of a danger to others these days because I am so much healthier. And other people are still more of a danger to me than I am to them. I still have a compromised immune system. We are all crawling with millions if not billions of microbes. You cannot even digest your food without them. Some microbes that are relatively benign in some populations are very dangerous to others. There are germs that create deadly infections in people with CF that are deemed benign for most normal people.

Peace.
You've picked some pretty poor examples to push your point.

Ebola outbreaks are very containable by Western medicine - the patients go into a quarantine ward, everyone observes fluid precautions religiously. It's never really made it out of Africa. In addition, traditional funeral rites in many areas exacerbated outbreaks, as they involved touching the dead.

Notification of partners is great for sexually transmitted diseases that are curable, but AIDS wasn't (and still technically isn't, even if sufferers can live long lives with current medical technology).

The plague killed somewhere between one and two thirds of Europe's population, so bringing it up as an example of successful cultural practices is rather baffling. Bubonic plague is entirely treatable by modern medicine, too.

Modern medicine fully endorses cooking food.

Malaria has been eradicated in the developed world by modern medicine, and modern technology has additionally made water safe to drink without needing to add ethanol to it.

> Diet, certain herbs and supplements, and an eye towards body chemistry can prevent viral infection and/or effectively treat it, something modern medicine generally does a poor job of.

Vaccination has done a great job of eradicating polio, smallpox, measles, and others. I eagerly await a list of diseases eradicated from the world via diet, herbs, and body chemistry.
I disagree. Both Facebook and MySpace tried to solve the problem of connecting people digitally, but that's where their likeness ends. They tried to solve that problem in fundamentally different ways.
There were some comments a couple days ago which might be helpful: http://news.ycombinator.com/item?id=2173482
I had a premature daughter, and I react with paternal anger whenever I see anybody threatening the health of my daughter. Do whatever you want to do to protect your own health, but stay away from my daughter.
Simple sorting questing for software developer interviews
Except for the schema and host parts (which are not part of robots.txt anyway) URLs are case sensitive (ref: RFC 3986, sections 6.2.2.2 and 6.2.3).

The problem here is that Microsoft servers respond to /search, /Search and /SeaRCh without distinction. They are all distinct URLs. If it was the intended behavior (stupid, but understandable, coming from Microsoft), then robots.txt should contain all variants in capitalization for each path. A better solution would be to force a 301 redirect to a canonical path, and have this path in robots.txt. Google would work as expected.

The original article is totally bogus. I can't imagine how it has over 90 votes.
It seems like you really need to keep up the index on the same %, but I guess dropping all the "designed for Ads only" sites would drop your number. I keep wondering if one of the shortening companies would come up with a search engine based on their links and some metric on repeats / reputation. Also, it might be work trying some touch setup with a person going good/bad for some % of your index.
There's a bit of a weird myth with robots.txt and the idea that it prevents pages from showing up in search engine indexes. Robots.txt means that the search engine cannot crawl the page - it can still include it in the search results if it sees enough sites linking to it. It can take a guess at what the page title can be, but there's usually no descriptive snippet because it's unable to see what's on the page.

If you don't want the page to be included in the index at all, you can use the meta noindex tag, by putting this in the head:

<meta name="robots" content="noindex" />

Pro tip: you need to also unblock that page from robots.txt - if Google isn't allowed to crawl the page, it can't see the meta noindex tag, which means it would stay indexed.
> It is a form of vaccination but many people who are anti-vax are mostly concerned with things like thimersol.

http://www.cdc.gov/vaccinesafety/Concerns/thimerosal/thimero...

> No, measles, mumps, and rubella (MMR) vaccines do not and never did contain thimerosal. Varicella (chickenpox), inactivated polio (IPV), and pneumococcal conjugate vaccines have also never contained thimerosal.

Additionally, you get more mercury from breathing in pollution from coal burning power plants than you ever will from vaccinations.
> "Search" != "search"

This is highly ridiculous. HTTP URLs are not mandated to be case-sensitive (though it's recommended), and clearly lots of sites use them in a case-insensitive way. Robots should either consider robots.txt in a case-insensitive way (even if I'm conscious that lot of them, including major ones, currently don't do that, which is precisely what I consider to be a problem, and this is supported by what happened here -- where google is risking to appear as a fool). The following article has perfectly good arguments in favor of case-insensitivity or even more clever handling: http://www.slicksurface.com/blog/2007-04/be-careful-robotstx...

Also; failing to respect conditions of use of a service because an automatic process is not safe enough is not a completely exonerating excuse for the operator of such process...
Palo Alto Store Project Active With Updated Design
Great link, I've definitely spent much longer than I expected reading these articles.
- Learn new stuff. But not every new thing. Yet another framework/language/library for xyz isn't usually worth it.

- Work with the smartest people you can.

- Strive for beautiful code.

- Keep simplifying.
To the best of my knowledge, Saudi Arabia is completely closed to archaeology due to the state religion (happy to be corrected on this one). If it ever does open up it will become a fantastic treasure trove of knowledge: all sites never before excavated (by scientists) and the location astride the cross-roads of all the old-world continents.
Manchester United Suffered Their First Premier League Defeat | SquidStop
Equally unethical sure (x=y), but differing opinions on whether that (x) is ethical or unethical. Personally, I see nothing wrong with it.
If you look at http://wap.bing.com/robots.txt, the URLs that Google is returning are actually all set to 'allow', not disallow.

It also looks like m.bing.com/robots.txt blocks /search while their actual URLs are /Search - I guess Googlebot treats robots.txt as case-sensitive.
I usually don't upvote pg's comments, because he could say "I like turtles." and get 1,000 upvotes, but pg's advice here is gold.

You should do this. You don't even need a publisher, you could approach your topic like Zed Shaw did with "Learn Python The Hard Way".

Or you could do a series of blog posts. You WILL get feedback and this will improve your skills.
A New "Moore's Law" for Neuroscience
I think vaccines should be mandatory for all children. If you don't vaccine your kid, CPS should take the kid away after multiple warnings, because you are an unfit parent and a danger to our community otherwise

They are in some places, and that is what happens unless you object on moral or religious grounds. I worked at a pediatric clinic, and on several occasions, we called CPS on parents that did not bring in their children for routine physicals and vaccinations.

Also, it's very difficult to get a child into the school system unless the child has been vaccinated.

/I moonlight as ER nurse, and I've done contract work at Chicago Children's, University of Chicago Children's, Fresno Children's and a satellite of Boston Children's.
I almost misread that as "the example is a synopsis of a television channel canceling the only show worth watching on their entire network to make room for more male drama shows such as wrestling"
There are 10x managers just like there are 10x engineers.
Science relies quite heavily on the concept of "settled science". The word used in science is "assumption". No scientist has the ability to prove everything from first principles in their work, they rely on a base of assumptions. If any of their assumptions get overturned, their own work gets overturned as well, so they choose their assumptions carefully. Sometimes that stack of assumptions can get pretty high and shaky, but science cannot operate without it.
Not all of them, but many of them. For example, how many cancer deaths could be avoided each year if the FDA banned the practice of spraying radioactive fertilizer on the tobacco leaves? It's thought that 90% of the cancer caused by tobacco comes from the radiation and not from the tobacco alone, which means that about 135,000 Americans are dying each year because of the FDA's malfeasance.
Doing a start-up is like going to college, especially for those who cant code. You put a ton of money into and either way the outcome should be positive; either it's a crazy success or it wasnt but you gained invaluable skills(coding & or online marketing), as well made some new friends by throwing yourself into local start-up communities. Don't isolate yourself; every town has a tech community.

Overall, his story mirrors mine; spent way over 100K & about 4 years on it. I dont regret it, as the journey was a lot of fun, a great learning experience and I met some great people/connections. We are even now are seeing others creating & having a shot at the same concept(which is neat to see).
The thing that bothers me about all this drama is that the actual offense Google wants everyone to be so worked up about is that Bing doesn't filter Google from its clickstream data.

Bing wrote code that works across the whole web. The whole web includes Google. As a result, Bing gets some info from Google. But they didn't get that info because they copied Google, they got it because they didn't filter Google out - or, said another way, because they ignored Google as someone they needed to special case for clickstream analysis.

I don't work in search, but the idea that you're supposed to special-case your competitors when writing general-purpose tools sounds an awful lot like a unilaterally recognized gentleman's agreement. If it's not illegal, and it doesn't hurt end users, why shouldn't it be considered fair game?

I also think it's odd that throughout this whole thing, nobody has really noticed that the only possible way Google could have spotted this issue is if they're keeping very close tabs on Bing's search results. It's another arbitrary line that Google seems to have unilaterally drawn: it's clearly fine to monitor your competitors results closely, which presumably is going to have an effect on your own results; it's only out of bounds when that effect is directly measurable.
High levels of population growth are unsustainable. There's a finite amount of land and other resources on the planet, and we've seen world population go from a billion in 1800 to about seven billion now. That level of growth cannot continue without triggering wars, famines, etc., and slowing it is a pretty admirable goal.

Infant mortality and subsistence farming in the developing world is one of the big causes for it. Raise the quality of living, prevent babies from dying, and people suddenly don't need to have 20 kids to subsistence farm a plot of desert.

It's only counterintuitive if you don't examine it closely.
Not sure how to ping? I'm on twitter @anymeansinc
Mystery of the mummy's Chinese travel ban