fix prettystr stackoverflow issue, and modify test cases

fix this issue

```
local t = {}
t.__index = t
prettystr(t) -- stack overflow
```