The BIOS/UEFI has an integrated temperature monitor for Windows 11, 10, and older versions. You can go there by booting up your computer and pressing a specific key (often F12, ESC, F2, or DEL) when prompted. The CPU temperature is shown prominently when you boot into BIOS or UEFI.