Fixes several swfdec AVM1 tests

jsshell has less stack than browser, so swfdec/test/trace/stack-overflow is still failuing, however the first commit will fix the infinite recursion issue.