dmoz support is now updated and re-integrated.
minor fix when indexing dmoz urls.
fix the search tabs for the dmoz directory search
box. allow more error types when spidering dmoz docs.
dmoz directory root page search box should
just search all sites in dmoz.
half way done fixing performance graph.
needs more work.
get performance graphing working again.
use absolute divs to draw the graph
instead of old gif plotter library.
remove old libplotter references
and files.
start using html div graph for
PageStatsdb.cpp now too.
still getting statsdb link to work. a little
better now.
fix "statsdb" graph so it seems to work
now.
fix "search in category" link.
fix potential problem of tons of points in
our statsdb div graph. use hashtable to dedup
points and save from printing out too many <div>
tags.
take out references to AdultBit.cpp,
since it is no longer used.
update the dirty word list. but we still
should remove tags, except maybe outlinks,
and detect the dirty words on what remains.
getting too many false positives in tags still.
speed up dirty word detection since we added a bunch
of new dirty words/phrases.
comment updates
ensure dmoz info valid when making titlerec
Merge branch 'master' of git@github.com:gigablast/open-source-search-engine
turn off log debug msg.
dirty word detector revisions. we need
a word-based function, isDirtyWord()
which IDs single words, bigrams and trigrams.
it'll be much faster than the current approach
and won't slow down when the list of dirty
words gets big. we then need isDirtyUrl() to
use the logic in Speller.cpp to split
a url up into composing words to run through
isDirtyWord().