duckduckgo/zeroclickinfo-fathead — pull request #3 by pardocz (2011-10-16 22:38:28+00:00)
Title: Initial implementation of Expatistan's Fathead

I've asked using open@duckduckgo.com how to include Expatistan's cost of living data into your zero-click results. Following the indications of Torsten Raudssus here is a pull request for it. Just to give a bit of context, expatistan.com is a crowdsourced cost of living index that I've created and that I run. It currently holds reliable information for 143 different cities around the world (and not so reliable info for 700 more). I think it would be great to have this information accumulated on cost of living show up in duckduckgo 0-click results for queries like "cost of living in Paris" or "how expensive is London" or "cost of living in New York compared to San Francisco". The dump I'm creating for Fathead will contain data only for the cities with reliable information.

I've used ruby to implement the parser, since that's what I am most familiar with. The parser is very simple, though, so I could port it easily to python or node.js is necessary.

I am not sure how the fetcher should work. I've based this initial one in some other one available in the repo (the c++ one, I think). I'll be happy to change it too if necessary (if I get indications from you guys on how it should actually behave, and where it should save the fetched dump).

The output of the parser may also not be totally correct, specially the first field ("title"). I assumed that this field would be the 'keyword' that would be used to match a search query with one of the entries in the parsed dump. Therefore, I've placed there the name of the city (or cities) for each entry, in English. In the case of comparisons of two cities, I've put in that first field the name of both cities, separated by semicolon. I don't know what convention do you have for multi-term "title". Again, I'll be happy to change it to whatever you guys use.