weppos/whois — pull request #312 by huyphan (2014-03-19 07:05:59+00:00)
Title: Smart parser

We experienced so many cases when the hosts are not supported by Ruby Whois but the body contents are similar to those of supported hosts.

This pull request offers a way to guess the parser based on the body of a Whois::Record::Part. It extracts the *keywords* from the body then compare them with existing signatures (i.e. list of keywords) generated from the txt files.

We added the rake task "signature:generate" to help generate the signatures and wrap them by a class in **/lib/whois/record/parser/signature.rb** file.

The guessing process is rather simple: it takes the signature with the most matching keywords. The matching ratio must be higher than 0.5 for the parser to be accepted, otherwise it returns Blank parser.

We made a quick test on a problematic domain:

Before the patch:

```
1.9.3-p484 :006 > Whois.whois("joker.net").admin_contact
=> nil
1.9.3-p484 :007 > Whois::Record::Parser.parser_for(Whois.whois('joker.net').parts[1]).class.name
=> "Whois::Record::Parser::Blank"
```
After the patch:

```
1.9.3-p484 :030 > Whois.whois('joker.net').admin_contact
=> #
1.9.3-p484 :031 > Whois::Record::Parser.parser_for(Whois.whois('joker.net').parts[1]).class.name
=> "Whois::Record::Parser::WhoisUdagNet"
```