In the world of the show, this happened several years ago, but Thomas has just gotten around to getting upset about it now.
Tom and his new lady love interest have a nice conversation in the pub, which gets awkward when she invites herself over to Downton.