Upgrade for Lucene 4.0.0 API compatibility

I have done some upgrade work so that it can now be used with the new Lucene 4.0.0 API and have also made some cleanup.
It was also tested to work properly with the Apache Solr 4.0.0