fix prettystr stackoverflow issue, and modify test cases

fix this issue

```
local t = {}
t.__index = t
prettystr(t) -- stack overflow
```
This one and the one below should be in another PR.