For a RootNode URL, the Gatherer will expand it into zero or more LeafNode URLs by recursively enumerating it in an access method-specific way. For HTTP, a RootNode URL is expanded by following the embedded HTML links to other URLs. There are several options that allow you to change these limits and otherwise enhance the Gatherer specification. After you have written the Gatherer configuration file, create a directory for the Gatherer and copy the configuration file there. It will also export the indexing information automatically to Brokers and other clients.
Compression is the default, but can be disabled with the -nocompress option.