Prepare for serious change.
A less naive version now seems to be working. The main difference is that now we have a fixed-size camera who moves around the universe. (Where as previously we just expanded the canvas indefinitely.