With the incorporation of advanced sensors into the operating room, machine learning provides an avenue to improve surgical outcomes. The diversity in disease, patients, and surgical techniques requires robust methods that are grounded in domain knowledge. In this talk, I present formulations for incorporating domain knowledge into modern surgical data science approaches for performance monitoring and assessment. I will also discuss future directions on the integration of sequential, spatial, and pathological context into models of surgery.