rewrite: when using lxml parser, just pass raw stream to lxml
without decoding. lxml parser expects to have raw bytes and will determine
encoding on its own. then serve back as utf-8 if no encoding specified.
should address #36
update to version 0.3.1
record reading: better handling of empty arc (or warc) records
for indexing, index empty/invalid length as '-' status code
for reading, serve as 204 no content.
ensure that StatusAndHeaders has a valid statusline when serving
if http content-length is valid,, limit stream to that content-length
as well as record content-length (whichever is smaller)
replace content-length when buffering
update bad.arc with empty record example
lxml: use lxml's parse interface instead of feed interface to allow
xml to handle decoding unicode data, better address #36
wombat update:
- support scheme-relative (//) urls
- override dom manipulation (appendChild, insertBefore, replaceChild)
- disable Worker() interface for now
minor tweaks: add default static_path for jinja,
remove unused import
update changes, add more tests
decompressingbufferedreader: default to 'gzip' decompression instead of
none. ChunkedDataReader also automatically attempts decompression, by default
Add tests to verify
add rewriter_handler, frame wrapper support!
refactor:
- move is_identity(), is_embed() to wburl from wbrequest
- add is_mainpage() predicate
- add create_template() to each J2TemplateView to create itself
- add HeadInsertView to create a reusable head insert for
RewriteContent
- add 'mp_' as modifier for frames mode to be used as possible
  modifier with HTMLRewriter
ChunkDataReader: if determined to be non-chunked, read full buffer
unchunked
support both frames and non-frames mode
add automatic framing when in framed mode
live rewriter: integrate handler with rewrite_live.py module,
clean up css, add unit and integration tests
clean up cli server now known as 'live-rewrite-server', which performs live rewrite using
iframe paradigm
fix: when reading response, only readline() if previous read()
was non-empty
add support for optional proxies (verify set to false for now)
html insert: add include_ts option to optionally not add timestamp
remove rfc3987 validation, was rejecting valid urls
add extract_referer_wburl_str() to extract WbUrl str, if any,
from the referrer. Use that for live_rewrite_handler to override
default referrer
live rewrite: use custom REL_REFERER field don't overrie HTTP_REFERER
if REL_REFERER not set, don't send any referrer
rewrite: wombat update 2.0! Using Object.defineProperty() to better
override .href and .hash properties when possible.
.href returns original url, but on assignment rewrites before redirecting
.hash proxies to location.hash
Also added:
- window.top -> window.WB_wombat_top
- document.referrer -> document.WB_wombat_referrer
- <source> html tag rewriting