zipnum: support for getting the location root path
Merge branch 'master' of github.com:internetarchive/ia-web-commons
CDXFile: don't include prev line in plain cdx bin search
WaybackURLKeyMaker: add ability to specify custom rewrite rules with startswith, regex and replace
Merge branch 'master' of github.com:internetarchive/ia-web-commons
make RewriteRule static
simple utility, sort of like grep but with unique options -kPOS (like sort -k) and -gGROUP (print matching group), that can be run from the command line to test java regular expressions
improve help message
zipnum: add optional cdxLinesPerBlock entry to ZipNumCluster
Merge branch 'master' of github.com:internetarchive/ia-web-commons
GeneralURIStreamFactory supports file:// protocol for local filesystem
IA canonicalizer rules supports option to not remove trailing slash (removing trailing slash is default behavior)
FieldSplitLine base class for a line split by delimiters
SummaryLine is a FieldSplitLine that is a cdx summary line
CDXLine also a FieldSplitLine
ZipNum system refactoring:
ZipNumIndex: main interface for loading zipnum content from a zipnum index
ZipNumCluster: an extended zipnum interface which supports a location and lastblocks file
cdx: Support FieldSplitLine having different names set by subclasses on init
CDX/ZipNum Refactoring
* Added AbstractSeekableLineReader, made SeekableLineReader interface
* Added support for getNthPage() to ZipNumIndex
* Sorted text file supports binary search with custom comparator
fix typo (line field not set!)
Add utility classes from cdx server:
* PublicCDXLine, a filtered CDXLine for public display
* UrlSurtRangeComputer for computing url and surt ranges for exact, prefix, host and domain ranges
* RegexFieldMatcher for regex matching against named fields
FIX: return original host not surted-version in host/domain query
CDX Server: Add PrefixFieldDeduper for deduping/collapsing query results