Fixed parsing of multibyte files at byte 1024

From ruby bug #6393, ruby commits r35531, 35534
Fixed rdoc -E

rdoc -E depended on the original file existing.  In the case it didn't
it would report the file as unparseable.  Original patch by Nobuyoshi
Nakada as ruby commit r35532.

The patch was updated to handle modeline support and rearrange the tests
to work properly.  Now can_parse will return a parser which will get
checked later by RDoc::Parser::for.

Ruby bug #6392
Ensure --extension is recorded in the parser list

Follow up commit to the rdoc -E fix
Skip Encoding-dependent test on 1.8
Fixed parsing of FFI's libffi ChangeLog

Time.parse doesn't handle arbitrary input well.  This caused a bug when
it couldn't figure out the format of the time properly.  Now we fall
back to splitting on two spaces to find the time when Time raises a
NoMethodError.

Fixes #165
RDoc can now link #=== methods

The METHOD_REGEXP_STR was missing === so it could not be linked.

Fixes #164
Allow [] in arguments for TomDoc

Fixes #167
Allow [] anywhere in tomdoc arguments
Increase version
Updated minitest dependency
Only trim known extensions from page name

Previously any extension on the file was trimmed, but this could give
confusing results.  Now RDoc only trims rb, rdoc, txt and md extensions.

Fixes ruby bug #7673
Do not document JavaScript files
Updated Manifest.txt
Fixed servlet for home and site directories

Due to missing tests RDoc::Servlet#store_for did not find the store for ri
documentation from site and home directories.  Tests and implementation
have been added.

Fixes #170
Fixed references to methods in the RDoc servlet

Visibility information (document_self) was not set properly for objects
loaded from an RDoc store.  This prevented references from appearing in
HTML output from the RDoc servlet (and likely links for ri output) since
the reference was found but #display? returned false.

Now visibility information is set upon load which prevents this bug.

Part of #171
Fixed debug message for 404 Not Found page in darkfish generator
Merge pull request #174 from gettalong/darkfish_debug_message_fix

Fixed debug message for 404 Not Found page in darkfish generator
Updated history for #174
Don't delete attribute data when re-saving a class

The RDoc store places attributes and instance methods in the same place
in the cache.  These are stored in the same place to make method lookup
with ri easier.

To determine which ri data files to delete from the store when saving an
update to a class ri subtracted the cached instance method names from
the instance method list but not the attribute names.

This caused RDoc to delete attribute data that was still present.

Now RDoc subtracts both the instance methods and attributes from the
cache list.  This prevents accidental deletion of ri data.

Fixes #171
Fixed root search index

The search index had filesystem directories instead of HTTP
directories so it would send you to the wrong page.  Now the HTTP
directories are given in the root search index.