The easiest way to tell if the game you're playing supports mouse and keyboard is to check its own settings menu. Most Xbox games that support mouse and keyboard, like Sea of Thieves, will have their own dedicated sections for customizing these peripherals.
If a game's options menu doesn't have sections for mouse and keyboard, it may still support them in a limited capacity. Though in most of these cases, it'll be just the keyboard that works in-game.