Context and transformers work - 2

Continued from #1100.

Not ready for merge -- opened to ask questions on the code and get some reviews/feedback.