Lazy load sources

Same as jhthorsen's original PR.
Rebased and fixed for changes since.

I'll be going through testing of this in the coming week.