html5lib/html5lib-python — pull request #26 by mfa (2013-04-24 12:43:00+00:00)
Title: Refactor sanitize_token for better subclassing 

For my application I need to clean all unallowed tokens.
At the moment the only way is to copy the whole sanitize_token method, remove a few lines and add one.
Splitting up the sanitize_token method would simplify this a lot and reduces code duplication (in my code).

Example usage:

```python
import html5lib
from html5lib import treebuilders, treewalkers, serializer, sanitizer

class MySanitizer(sanitizer.HTMLSanitizer):
# reduce tokens to only a few
acceptable_elements = ['b', 'br', 'center', 'em', 'h3', 'h4', 'h5', 'h6',
'i', 'li', 'ol', 'p', 'span', 'strike', 'strong',
'tt', 'ul']

allowed_elements = acceptable_elements

def unallowed_token(self, token):
# remove all unallowed tokens
return ""

def sanitize(input):
p = html5lib.HTMLParser(tokenizer=MySanitizer, tree=treebuilders.getTreeBuilder("dom"))
dom_tree = p.parseFragment(input)
walker = treewalkers.getTreeWalker("dom")
stream = walker(dom_tree)

s = serializer.htmlserializer.HTMLSerializer(omit_optional_tags=False,
quote_attr_values=True)
return u"".join(s.serialize(stream))
```