Check if we're doing any deprecated things

We should turn all deprecation warnings *on* when running the tests (ideally by default, but that's probably fungible), so that we can see if we're doing anything that will break in future versions so that things like #1945 are less of a surprise.