Topic/patmat inference prep

The first three commits of #2894 which are ready to merge.