No, you do not have to see the queen - you should see evidence of her being in the hive - eggs, larvae etc.