Yes, you should always check the temperature with a thermometer! Do not rely on the pop up timers. You can use it as an indicator of when to check the temperature. I have had the pop up time pop up many times before the turkey was up to temperature.