Backport caching jars

In addition, added synchronization around `getOrElse`.

I will cherry-pick that commit on top of master as soon as this one is reviewed.