Before revision:  We adapt common representation learning techniques used in other domains and find that representations inspired by language models enable a 3.5\% mean improvement in AUROC on five clinical outcomes compared to standard baselines, with the average improvement rising to 19\% when only a small number of patients are available for training a prediction modelfor a given clinical outcome .
After revision:  We adapt common representation learning techniques used in other domains and find that representations inspired by language models enable a 3.5\% mean improvement in AUROC on five clinical outcomes compared to standard baselines, with the average improvement rising to 19\% when only a small number of patient records are available for training a prediction modelfor a given clinical outcome .