Revert "Remove ALL world references for world unloading."

This reverts commit ff59560c9471f81210c36d0435ffd823e03a7eb6.