A pattern tree-based approach to learning URL normalization rules
Collection statistics for fast duplicate document detection