add test for add_val(..) and test for super long tokens

add_val: test each configuration of input variables
long tokens: make sure tok_stream correctly gets a token
             which spans 3 or more seperate input buffers.

also in tok_stream.c#add_val: if *len < 1 consider *base
to be empty
added documentation comments to tok_stream.h
add license to _info