melllvar/grr — pull request #16 by cardre (2013-07-11 09:46:25+00:00)
Title: Fix atom parser articles with missing link

Some Atom feeds are not adhering to the guidelines for publication of an article's link. As a result, no links are successfully extracted for the entire feed and will therefore fail on db insertion during staging because of a null link_url column. This means that no articles are being crawled and the feed fails every time. See https://www.sparkfun.com/feeds/news as an example. In this particular feed, each entry's link does not have a rel="alternate" attribute set.

This fix does an additional check if the link_url was not able to be extracted. It will then try any link element and use the first one found. This allows the feed above to have its articles successfully included.