duckduckgo/zeroclickinfo-fathead — pull request #36 by flaming-toast (2012-08-14 00:26:09+00:00)
Title: UNIX man-page plugin (fathead)

Hi there! :D

I've been working on the unix man-page idea:
https://duckduckhack.uservoice.com/forums/5168-plugins/suggestions/2678996-manual-pages-return-unix-like-man-info

This'll be my initial code check-in. There's probably still a few bugs here and there, but for the most part, I believe my code does what it's supposed to do.The output file follows the programming data file format.

I'd also LOVE to have some feedback and advice. In particular, I'd like some ideas for fetching data faster.

I CAN fetch the data properly, and the code can parse it properly but fetching takes a LONG time because the number of available man pages is MASSIVE.
I haven't found any compressed archives of all the man pages (the ones that do exist omit Section 1!), and I am unable to to grab compressed html files from my data source because of their robots.txt file (they do openly allow DuckDuckBot to crawl the site though, so maybe you guys will be have better luck testing this than I do).

Anyway, try the plugin out and let me know your thoughts! Thank you!!